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(57) Abstract 

A computer-controlled microscope (16) captures a plurality of images at a low magnification. These images arc tiled to create a 
reconstrected image (24) of an entire specimen. The user selects one or more areas of the reconstructed image. Hie computer-controlled 
microscope (16) then captures a plurality of high resolution images (26) of the selected areas for display to the user. 
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METHOD AND APPARATUS FOR ACQUIRING 
AND RECONSTRUCTING MAGNIFIED SPECIMEN IMAGES 
FROM A COMPUTER -CONTROLLED MICROSCOPE 

Field of the Invention 
5 This invention relates to a method of and 

apparatus for acquiring and recording digital images of 
an optical image viewed through a computer-controlled 
automated microscope and also, to using the latter in a 
quantitative analysis of plant or biological specimens. 

10 Background of the Invention 

In the image analysis and quantification of DNA 
from tissue sections as disclosed in United States Patent 
4,741,031, and also especially in the immunohisto- 
chemistry assays on the kinds of cell analysis systems 

15 disclosed in United States Patents 5,086,476; 5,202,931; 
and 5,252,487 issued to Bacus, there is a problem of 
first locating the cancer regions for analysis under low 
power and then remembering them when performing the 
analysis under higher power. There is a need, a 

20 requirement to image and digitally record an object in a 
relatively flat plane at high resolution/magnification. 
Today, it is impractical to construct an optical image 
sensor large enough to cover the entire image area e.g., 
of a specimen on a microscope slide, at the required 

25 resolution. This is because lens size and 

resolution/magnification issues limit the size of the 
field of view of magnified objects and their resulting 
images . Viewing through a microscope is akin to viewing 
through a periscope in that one sees a very small field 

30 of view even at low magnifications, such as 1-25X. A 
pathologist using a microscope often scans a slide to 
obtain in his mind an overall view or sense of what 
constitutes the specimen and he remembers the general 
locations of the diagnostically significant, small pieces 

35 of the specimen. Usually, these are the diseased areas, 
such as malignant or potentially malignant portions of 
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the specimen. To obtain higher resolution and 
magnification of these suspicious portions, the 
pathologist switches to a higher magnification objective 
lens but then the field of view becomes much smaller 
5 again. Often, the pathologist switches back and forth 
between the lower magnification, larger field of view 
objective lens to orient himself relative to the specimen 
and the high magnification, smaller field of view to 
obtain the detailed, high resolution view of the 
10 suspicious area on the specimen. Thus, the user never 
receives a magnified, condensed overall view of the 
specimen or a portion of the specimen but must remember 
the series of views taken at low magnification. 
Likewise, at high resolutions and high magnifications 
the user never receives or views a collection of adjacent 
images but must interrelate these successive images in 
the user's mind. 

Stated differently, the problem is that once the 
microscope is set up for quantitation by image analysis 
under, e.g. 40x. where all of the diaphragms are set and 
light adjusted, etc., if the operator needs to move to 
another tissue area, it is first desirable to locate it 
at e.g. lOx. in fact, often regions can only be located 
at this power. In order to do so however, all of the 
settings (diaphragms, light levels, wavelengths of light 
etc.) have to be changed to view the tissue at this 
magnification. Currently, there is no way to ensure that 
one could go back to the settings at the previous 40x 
magnification and continue on with the quantitative image 
analysis of that same specimen. This necessitates 
finding those areas under 40x, without changing 
objectives, which is very slow and time-consuming, and 
often important cancer areas can be missed. 

Also, another problem with tissue analysis at 
Its current state-of-the-art, is that it is not 
completely automated, for example, with regard to finding 
structural regions such as glands, basal layers or other 
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important diagnostic regions. However, as set forth in 
my co-pending patent application Serial No. 701,974, 
filed August 23, 1996, if these regions are located, 
important very sensitive diagnostic measurements can be 
5 performed, which patent application is incorporated by 
reference as if fully reproduced herein. For example, as 
disclosed in the aforesaid patent application, assays are 
made of a variety of tissue types, both human and animal 
for analysis of neoplasia in tissue, for pre-invasive 

10 cancer in tissue, and the effects on the tissue of 

chemopreventive agents. A quantitative analysis by image 
processing techniques is performed on tissue types, 
having various architectural features, such as breast 
tissue, colon tissue, prostate tissue, esophageal tissue, 

15 skin tissue, cervix tissue, etc. These tissues have 
different morphologies, and they undergo different 
neoplasias usually resulting from a cellular mutation, as 
may be enhanced by a carcinogen, or resulting from a 
cellular proliferation rate enhanced by hormones, growth 

20 factors, or other inducers of abnormal tissue growth. 
Often it is desired to quantify small changes in the 
neoplasia when it is incipient . or through a series of 
analyses performed at close time intervals to measure 
whether the neoplasia progression is increasing or has 

25 been slowed, stopped or regressed - 

Usually, the tissue specimens are cut to expose 
the basal layer for review under the microscope . 
Typically, the quantitative measurements are performed at 
40x to obtain 100 to 4 00 tissue images. The 40x 

30 objective provides a narrow field of view of a very small 
portion of the entire basal layer. Often, the basal 
layer is somewhat elongated and generally linear such as 
a basal layer in a rat esophagus; and the analysis of the 
basal layer requires examining it along its length. The 

35 basal layer in a mouse colon is more in the form of an 

irregular, circular shape; and the analysis of this basal 
layer requires traveling about this circular shape. In 
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breast tissue samples, suspected tumor areas may be at 
widely-spaced points in the stained tissue; and one wants 
to be able to navigate and travel to these specific 
suspected areas and to do the 4 Ox analysis at these areas 
in an efficient manner. There is a need to allow an 
experienced operator to interact with the analysis to 
locate and identify such regions in an interactive 
manner. Especially, an interactive manner that would be 
familiar and consistent with the use of a microscope 
manually, with high power magnification and low power 
magnification simultaneously available, but performed on 
a computer terminal. Such a level of interaction is 
different than the interaction with the system disclosed 
in the above-listed Bacus Patents. There is a need to 
take the level of interaction to a higher level and let 
each component, the human and the computer, perform the 
part that it does best, in the most cost-effective 
manner . 

The current state of archiving the digital 
images achieved through a microscope is often by having 
photographs or by video tapes. The photographs are 
difficult to use as is a video tape particularly when the 
user wants to move rapidly back and forth between various 
images and to scroll through various adjacent parts of 
the specimen image. Further, current archival methods 
lack an overall macro image of the specimen, which allows 
the user to know exactly where the particular high 
resolution view is from when the user is making an 
analysis of the high resolution image. 

While digitized images can be stored 
magnetically or otherwise digitized and recorded on 
various recording mediums, no current archival system 
allows the user to toggle back and forth between high 
magnification images and low magnification images or 
between various images at different magnifications such 
as that achieved by a pathologist switching microscope 
objective lenses in real time to get the macro and micro 
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images from the same location on the specimen. 
Heretofore, the practice of pathology has been relatively 
limited to the use of microscopes and to the pathologist 
having to use the microscope to review the particular 
5 specimen - 

There is a need for a dynamic system whereby one 
or more or several pathologists, including a consulting 
pathologist, may view the same area simultaneously and 
interact with one another either in diagnosis or in 

10 analysis. Also, it would be best if the images from the 
specimen could be stored so that a pathologist could 
easily examine the images at his leisure using an 
intranet or Internet browser at a later date merely by 
accessing the particular web site where the images are 

15 located. 

A similar problem exists on the Internet or 
intranet where a pathologist may receive a single field 
of view magnified image taken from a specimen over the 
Internet or the intranet on his browser. The pathologist 

20 must be provided with explanations to coordinate the high 
resolution view with the lower resolution view. The 
number of views available to the pathologist is very 
limited, and the pathologist is unable to select other 
views or to scroll to neighboring views at the areas that 

25 are most interesting to the pathologist. 

There are available on the market computer- 
controlled, automated microscopes such as those sold by 
Carl Zeiss, Inc., Thornwood, N.J., under the name 
Axioplan 2 for taking photographic images of a specimen 

30 in the microscopic field of view. Those particular 

microscopes have computer-controlled and automatically 
adjusted subsystems, such as an illumination subsystem, a 
focusing subsystem, a diaphragm or optical stops 
subsystem, an objective lens subsystem, or a filtering 

3 5 subsystem. As an operator selects changes from one 

objective lens, such as "one providing low magnification, 
e.g., 4x, to a higher magnification, e.g., 40x, the 
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computer-automated system will turn the lens turret to 
switch in the high magnification automatically and 
adjusts the lens and also automatically adjusts the 
illumination to eliminate glare and to provide the proper 
5 light illumination including light density. Further the 
focus is adjusted, and the proper diaphragm openings 'are 
automatically reset. Thus, the computer-controlled 
automated subsystems automatically reset to values stored 
and predetermined for each selected objective lens and 
10 the analysis being done. 

Those particular microscopes can be used to view 
varxous objects or specimens, but are most typically used 
to view and to take still photographs of biological 
specimens, such as tissues and cells. Those particular 
microscopes lack a computer- controlled X and Y stage for 
translating a specimen -carrying slide with respect to the 
field of view of the selected objective lens. Currently 
pathologists and others who use such microscopes want to' 
view the specimen images in full color or in enhanced 
colors using fluorescent illumination and/or 
monochromatic images using the automated filter 
subsystems on the microscopes. Currently trained 
pathologists or clinicians are accustomed to manually 
adjust and have a microscope available to them to view 
larger areas of the specimen at low magnification, and 
then to momentarily switch in a new higher magnification 
lens to obtain a more highly magnified image of a portion 
of the specimen viewed at low magnification. 
Pathologists and those working in this area have created 
in themselves a desire to view suspect tissue as though 
they were viewing the specimen tissue through a 
microscope and appear to resist analysis systems that do 
not provide them this ability. 

The microscopic field of view reduces very 
substantially as the magnification increases. The skill 
level of the clinician and/or pathologist is important to 
locate viewing the most suspicious areas or points of 
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interest on the specimen. Sometimes, a technician will 
do a first assay and analysis. A pathologist will return 
to the selected points of interest or other points of 
interest for review and analysis . One concern with 
5 respect to a quantitative analysis of breast cancer 

tissue or prostate biopsy tissue samples to pap smears or 
other tests for various cancers or the like is that a 
particularly suspicious point in the tissue may be 
overlooked and missed during the visual assay or for 

10 selection for an automated review analysis. _ When 

observing at high magnifications', the field of view is 
limited to very small area of the specimen. Hence, the 
observer has difficulty in knowing and remembering the 
actual, exact location of this small periscopic view 

15 within the very large whole specimen. 

Often, also the problem is finding or locating 
the tissue or cells for view at high magnification so 
that artifacts and/or blank spaces on the slide are not 
viewed. A number of approaches have been proposed to 

20 prescreen and locate by an X and Y address the cells or 
small points of interest from a very large number of 
potential points of interest,. 

There are currently available commercial 
services for prescreening pap smears where one can mail 

25 in slides and the service will do a microscopic prescan 
at high magnification for suspected or suspicious areas 
of interest which are marked and given address locations, 
and also a video tape of the slide specimen is returned 
by this service to the sender. The sender then reviews 

3 0 the areas of interest located during the prescreening 
and/or the video tape to complete the analysis* 

In an attempt to locate and allow review of 
specified points of interest, U.S. Patent No. 5,428,690 
to Bacus discloses a system for prescreening of a field 

35 of cells on a specimen slide at low magnification before 
the viewer. When seeing a point of interest to be viewed 
at high magnification, the viewer will operate a switch 
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or the like to select and record the address of these 
selected prescreened points of interest. Later, these 
prescreened points of interest are then brought into 
position to be analyzed at high magnification. This 
' system is fairly or too slow for many uses. 

A very expensive system is currently in use in 
which a pathologist located at a diagnostic center is 
able to make a diagnostic opinion with respect to 
specimens under a microscope at a remote center. The 
pathologist at the diagnostic center manipulates robotic 
controls to send telepathy signals via a special, 
dedicated satellite or other large bandwidth channel to 
control the microscope at the remote site in 
approximately real time. The pathologist then can 
maneuver the remote microscope to shift the microscope's 
field of view and to send, by telepathy, a highly 
magnified, very small image back to the pathologist. 
This system requires each subscriber to have a special 
microscope operable by manipulation of the robotic 
controls at the diagnostic center and a dedicated or 
large bandwidth channel to convey real time video signals 
and hence results in a very high cost for the assay being 
done . To assist the pathologist in staying within the 
specimen at the remote site, a peripheral edge or map of 
the specimen is made using a second video camera and a 
light box or by using computerized scanning equipment to 
trace the outline or peripheral edge of the specimen 
boundaries. A small circle of light is displayed within 
the map of the specimen so that the pathologist at the 
diagnostic center knows the location of the field of view 
of the highly magnified image within the specimen m a 
sense, the pathologist is operating in real time in the 
same way that he would use his own microscope at his 
diagnostic center except for a slight transmission delay 
used to transmit the video signals of the highly 
magnified image over large bandwidth channel. Although 
the pathologist has a small map or peripheral outline of 
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the specimen, the pathologist's field of view of the 
actual specimen is only the small circle of view that is 
coming through the microscope objective lens. This does 
not help the pathologist locate suspicious areas of 
5 interest as in a prescreening of the entire tissue. The 
pathologist may switch to the lowest magnification to get 
the largest field of view of a small section of the 
specimen, but he never views the entire specimen at any 
magnification. Also, there is no image analysis 

10. quantitative testing from the received images at the 

diagnostic center; and no quantitative assaying is done 
with these images at the diagnostic center. 

There is a particular interest today in using 
the Internet system because it is so readily accessible 

15 by users at a low cost and using a computer and viewing 
screen connected to the computer. One problem with 
trying to do any transmission of digitized, microscopic, 
highly magnified images over the Internet is that the 
bandwidth is too narrow to accommodate the tremendous 

20 amount of stored data which needs to be transmitted. 
There is a need for a system which would allow a 
pathologist or another person, to be able to perform 
tissue analysis or quantitative assays using a standard 
computer terminal from a location remote from the 

25 automated microscope system that digitized the optical 
images . 

Sxunmarv of the Invention 
In accordance with the present invention, a 
person such as a pathologist at a computer terminal may 

30 view on a computer screen or monitor digitized images of 
a microscope specimen at different magnifications as 
selected by the person. Further, the person may receive 
on the screen a low magnification, reconstructed image of 
the entire specimen to aid the person in interactively 

35 selecting points of interest on the specimen, such as 
along a basal layer of a tissue specimen. 
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More specifically, and in accordance with the 
present invention, the microscope's small field of view 
limitation of a specimen is overcome by providing to the 
viewer a reconstructed, digitized and magnified image of 
the entire specimen (or a very large portion of the 
specimen) for performing a visual analysis of the entire 
tissue in full color to aid in the selection of points of 
interest to be viewed at a higher magnification and 
higher resolution. This is achieved by acquiring and 
storing a large number of low magnification images of the 
specimen through a microscopic scanning system, e.g., 35 
image tiles of the specimen at 1.25X, and then assembling 
and coordinating these stored tiled images to form an 
overall, low magnified image of the specimen, i.e., a 
macro image of the specimen. Preferably, the digitized, 
stored macro image is reduced in size by a software 
system to a smaller size, e.g., a 1/4 size image and it 
is this reduced in size macro tiled image that is 
displayed on a local screen or is sent over a low band 
width or a high band width channel to a remote screen. 
The pathologist not only does not need to have others do 
a slow laborious prescreening to locate suspicious areas 
for analysis or for viewing at high magnification, he can 
use his own experiences to enable him to go directly to 
the most suspicious areas which he sees on the macro 
image. He can, on a priority basis, do the most 
suspicious area first, followed by lower priority areas 
of interest. 

In accordance with the present invention, there 
is provided a new and improved microscopic display system 
that displays the low magnification composite image of 
the specimen to allow the user to view and to 
interactively select points of interest, each of which 
may be displayed at high magnification and resolution. 
This is achieved by providing the user with a marker, 
such as a cursor or the like, to select the defined area 
of interest from the stored, macro image; and to acquire 
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reproduced, spatially adjacent high resolution, digitized 
images of the selected area of interest . More 
specifically, the specimen, when it was first scanned at 
low magnification to provide a macro view of the 
5 specimen, the addresses or locations of the tile images 
and/or pixels for the composite image were acquired. 
Therefore, any selected region of interest in the macro 
image has locations to which the microscopic stage may be 
automatically repositioned under a changed, higher 

10 magnification lens to acquire higher magnification, 

digitized image tiles that can be assembled into a micro 
image. Herein, both the macro and micro images are 
formed of adjacent digitized image tiles which have been 
coordinated to reproduce spatially the original image 

15 that was scanned. 

It is the high magnification images, usually at 
40x, that were analyzed using image processing techniques 
as disclosed in the aforesaid patent application, to 
provide an assay or numerical histological data for the 

20 specimen. 

In accordance with the preferred embodiment of 
the invention, the pathologist may desire to see a larger 
region for analysis at high resolution than can be 
accommodated at this magnification on his high 

25 magnification viewing screen. The pathologist can 

quickly change the view on his screen to view adjacent, 
highly magnified, digitized image tiles at this high 
resolution by scrolling up or down or right to left to 
shift these digitized, adjacent image tiles into view on 

30 the screen. Thus, even at a higher magnification of a 
. region the pathologist is able to obtain a much larger 
view than the small field for the objective lens in use 
of adjacent tissue or cells to give him a broader, 
overall perspective of what is happening or what has 

35 happened in specific section of a specimen. For 
instance, a pathologist may want to see at high 
magnification and to assay at this high magnification, a 
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suspicious area, the pathologist can draw a mark about 
the area and cause it to then be assayed and displayed. 

By having displayed on a low magnification 
screen or split screen of the full composite area, by 
having the high magnification region being marked on the 
low image screen, and by having the region being scrolled 
to view adjacent high magnification images on the high 
magnification screen, the pathologist has available 
information to guide him in the sense of helping to 
navigate intelligently within the specimen in his search 
for cancerous tissue or the like for further inspection 
or measurement of malignant characteristics. Often, when 
restricted to a field of view of an objective microscope, 
the pathologist has a difficult time, in the words of a 
cliche, of seeing the forest for the trees. In the 
present invention, the pathologist has a full, magnified, 
reduced in size specimen view with the higher image area' 
marked thereon to guide him and to let him see the 
forest . He can also see a region of the forest at higher 
magnification by scrolling adjacent tree images onto the 
high magnification screen. 

In accordance with a further aspect of the 
invention, the user may elect to view an intermediate 
magnification and resolution. In the preferred system, 
at least three magnification images are acquired and 
stored as digitized images. The intermediate resolution 
images may be acquired by either switching automatically 
to a new objective lens and acquiring new digitized image 
tiles at the intermediate magnification or by using 
software to reconstruct a new intermediate digitized 
image from the existing high and low magnification, 
digitized images. 

The preferred, low magnification image, which is 
reduced in size, can be transmitted over narrow band 
35 width channels such as a local area network or over the 
Internet through various servers and computers. 
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Likewise, the reconstructed, high magnification images 
can be transmitted over such narrow band width channels . 

The preferred microscope is fully computer 
controlled, a pathologist or other person having a split 
5 screen computer such as a PC, can be connected to the 

microscope and operate it from a remote location with the 
computer microscope system to digitize and construct the 
macro image. Likewise, the pathologist can navigate to 
points of interest and with the computer microscope 
10 system to digitize and construct the desired micro 

images. With the present invention, there is no need for 
a specialized microscope at each remote location nor for 
a broad band channel to send video signals in real time 
between the diagnostic center and the remote location. 

15 Brief Description of the Drawings 

FIG. 1 is a screen view of a system embodying 
the present invention showing a low magnification image 
of a specimen on a microscope slide in one window, a high 
magnification image of a portion of the low magnification 

20 image selected by a region marker and a control window; 

FIG. 2 is a view of a display screen of the 
apparatus embodying the present invention showing the 
control window a low magnification window having a 
plurality of high magnification micro image regions 

25 delineated therein and a high magnification window 
including one or more of the micro image regions; 

FIG. 3 is a view similar to FIG. 2 including the 
control window but also including a low magnification 
region from the slide showing regions marked by a 

30 histology grade or structure through automatic analysis 
of tissue and a high magnification window showing 
markings related to the grading or histology grade 
yielded by the automatic analysis of tissue in 
combination with a window showing a numerical score; 

35 FIG. 4 is a block diagram of the apparatus 

embodying the present invention; 
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FIG. 5 is a block diagram of a portion of the 
apparatus shown in FIG. 4 showing details of a mechanical 
arrangement of a microscope; 

FIG. 6 is a flow diagram related to operation of 
5 the apparatus; 

FIG. 7 is a flow diagram of details of one of 
the steps in FIG. 6; 

FIG. 8 is a display screen showing control 
parameters to be manipulated thereon; 

FIG- 9 is a flow chart for a region outlying 

routine; 

FIG. 10 is a flow chart for a scanning and 
analyzing routine; 

FIG. 11 is a schematic showing of the limits of 
15 travel of the microscope stage with respect to the image 
tiles; 

FIG. IIA is a perspective view of the microscope 
stage and stepper motors and encoders providing a closed 
loop drive for the motors; 

PIG- 12 is a block diagram of a networked system 
allowing multiple workstations to obtain access to 
diagnostic image information and to manipulate such 
information locally at each workstation; 

FIG. 12A is a view of the system described in 
25 connection with FIG. 5; 

FIG. 13 is a block diagram of a remote networked 
system for distributing and accessing diagnostic images 
and data through a hypertext transport protocol based 
server directly or over a packet network; 

FIG- 14 is a view of a low magnification, 
reconstructed image from a basal layer of rat esophagus; 

FIG. 14A is a view of a high magnification, 
reconstructed image from a selected point of interest 
from FIG. 14; 

FIG. 15 is a view of a low magnification image 
of a mouse colon having a basal layer; 
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FIG. ISA is a view of a reconstructed macro 
image of a mouse colon; 

FIG. 16 is a schematic view of an analysis from 
regions of a basal layer; 
5 FIG. 16A is a schematic view of an analysis to 

provide to a Z score; and 

FIG. 17 is a schematic view showing texture 
analysis tests for regions. 



Detailed DeBcription of the Preferred Embodiment 

10 Referring now to the drawings, and especially to 

Figs. 4 and 5, apparatus for synthesizing low 
magnification and high magnification microscopic images 
is shown therein and generally identified by reference 
numeral 10. The system includes a computer 12 which is a 

15 dual Pentium Pro personal computer in combination with a 
Hitachi HV-C20 video camera 14 associated with a Zeiss 
Axioplan 2 microscope IS . The computer system 12 is able 
to receive signals from the camera 14 which captures 
light from the microscope 16 having a microscope slide 18 

20 positioned on an LUDL encoded motorized stage 20. The 
encoded motorized stage 20 includes a MAC 2000 stage 
controller for controlling the stage in response to the 
computer 12 . A microscope slide 18 includes a biological 
specimen 21 which is to be viewed by the microscope and 

25 whose image is to be digitized both at low magnification 
and at high magnification as selected by a user. The low 
magnification digitized image is then displayed on a 21 
inch liyama video display monitor 22 having resolution of 
1600 by 1200 to provide display screens of the type shown 

30 in Figs. 1 through 3 including a low magnification image 
24, for instance, at 1.25 power, a high magnification 
image 26, for instance at 40 power and a control window 
or image 28 . The low magnification image may have 
identified therein a region 30 which is reproduced at 

35 high magnification in high magnification screen or window 
26 so that a pathologist or other operator of the system 
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10 



15 



20 



25 



30 



35 



can review architectural regions of interest in low 
magnification image 24 and simultaneously view them in 
high magnification in the high magnification screen or 
window 26 to determine whether the cells forming a 
portion of the architectural feature need be examined 
further for cancer or the like or not. 

The con^juter 10 is constructed around a PCI 
system bus 40 and has a first Pentium Pro microprocessor 
42 and a second pentium pro microprocessor 44 connected 
thereto. The system bus 40 has connected to it a PCI bus 
50 and an ISA bus 52. The PCI bus 50 has a SCSI 
controller 60 connected thereto to send and receive 
information from a hard disk 62 . The hard disk 62 also 
is coupled in daisy chain SCSI fashion to a high capacity 
removal disk and to a CD Rom drive 66. The hard disks 62 
contains the programs for operating the system for 
controlling the microscope 16 and for processing the 
images as well as for doing a quantitative analysis of 
the selected portions of the histological specimens being 
viewed on the slide 18. The system bus 40 also has 
connected to it a random access memory 70 within which 
portions of the program being executed are stored as well 
as a read only memory 72 for holding a bootstrap loader 
as well as portions of the basic input/output operating 
system. A floppy disk controller 74 is coupled to the 
system bus 40 and has connected to it a floppy disk drive 
76 for reading and writing information to a floppy disk 
as appropriate. A mouse controller 80 is coupled to the 
system bus and has a mouse 82 which operates as a 
pointing device for controlling manipulations on the 
screen 22 and within the windows 24, 26 and 28. A 
keyboard controller 90 is connected to the system bus and 
has a keyboard 92 connected thereto. The keyboard 92 may 
be used to send and receive alpha numeric signals to 
other portions of the computer. An audio controller lOO 
has a plurality of speakers 102 and a microphone 104 
connected thereto for audio input and output and is 
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coupled to the system bus 40. A network interface, such 
as a network interface card 104, is connected to the 
system bus and can provide signals via a channel 106 to 
other portions of a network or internet to which the 
5 system may be connected. Likewise, signals can be sent 
out of the system through a modem 110 connected to the 
ISA bus 52 and may be sent via a channel 112, for 
instance, to the internet. A printer 116 is connected 
via a parallel I/O controller 118 to the system bus in 

10 order to provide printouts as appropriate of screens and 
other information as it is generated. A serial I/O 
controller 122 is connected to the system bus and has 
connected to it a camera controller 124 which is coupled 
to CCD sensors 126 in the cameras. The CCD sensors 126 

15 supply pixel or image signals representative of what is 
found on the slide 18 to an Epix pixci image acquisition 
controller 130 coupled to the PCI bus 50. 

The microscope 16 includes a base 140 having a 
stage 20 positioned thereon as well as an objective 

20 turret 142 having a plurality of objectives 144, 146 and 
148 thereon. The objective 144, for instance, may be of 
1.25X objective. The objective 146 may be a 20X 
objective. The objective 14 8 may be a 4 OX objective. 
Signals from the camera sensors and controller are 

25 supplied over a bus 128 to the image acquisition system 
where they are digitized and supplied to the PCI bus for 
storage in RAM or for backing storage on the hard disk 
62. 

When a specimen is on the slide 18 the stage 20 
30 may be manipulated under the control of the computer 

through a stage controller 160 coupled to the serial I/O 
controller 122. Likewise, a microscope controller 162 
controls aspects of the microscope such as the 
illumination, the color temperature or spectral output of 
35 a lamp 16 8 and the like. For instance, in normal 
operation, when a specimen is placed on the slide, 
specimen slide 18 is placed on the stage 20 in a step 
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200, as shovm in FIG. 6, the processors 42 or 44 send a 
command through the system bus to cause the serial I/O 
controller 122 to signal the microscope controller to 
change magnification to 1.25X in a step 202. This is 
5 done by rotating the objective turret of the Axioplan 2 
microscope to select the objective 144. Likewise, the 
controller sets the color temperature of the lamp 168, 
sets a pair of neutral density filter wheels 170 and 172 
and sets a field diaphragm 174 for the correct 
10 illumination. A condenser diaphragm 176 is also 
controlled and a color filter wheel 180 may also be 
controlled to apply the appropriate filter color to the 
CCD censors 126 in the camera. The entire slide is then 
scanned in a step 204. The images are tiled and melded 
together into the overall image 24 supplied on the screen 
22 to provide the operator in the step 206 with a 
visually inspectable macro image of relevant regions of 
the slide of interest . 

In order to provide the magnified image, the 
mouse may be moved to identify a marker segment or region 
which, for instance, may be a rectangular region (as 
shown as 30 in FIG. 1) which will cause the microscope to 
change magnification as at step 208 to 4x, 20x, 40x, 
etc., by rotating the turret to bring the appropriate 
25 objective lens system into viewing position. 

Next the user, in a step 209a, uses the mouse to 
select the region on the macro image in order to select 
the micro image to be viewed on the screen 22. in a step 
209b a test is made to determine whether the user has 
commanded continued inspection. If the user has, a test 
is made in a step 209c to determine if the magnification 
is to be changed by changing the selected objective. In 
the event the magnification is to be changed control is 
transferred to the step 208. If the magnification is to 
remain unchanged control is transferred to the step 209a. 
In the event inspection is not to continue the region 
selected is outlined for higher magnification scan in a 
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step 209d. In a step 209e, a command may be received to 
scan or acquire the higher magnification image for 
display in screen 26 . The image may then be archived for 
later analysis, displayed or analyzed immediately. 
5 In order to perform the magnification called for 

in step 208, the overall illumination and control of the 
microscope will be controlled so that in a step 210 the 
objective turret 142 will be rotated to place the higher 
power objective above the slide 18. In a step 212 

10 voltage to the lamp will be changed to adjust the lamp 
168 to provide the proper illumination and color 
temperature as predetermined for the selected objective. 
In a step 214, the condenser diaphragm 176 will have its 
opening selected as appropriate to provide the proper 

15 illumination for that objective. In a step 216, the 

filter turret 180 will select the proper light wavelength 
filter to be supplied to the camera sensors. For 
instance, a red, blue or green filter, as appropriate, 
particularly if the specimen has been stained. In a step 

20 218 the field diaphragm 174 will have its opening 

changed. In a step 220 the neutral density filter wheel 
170 will select a neutral density filter and in a step 
222 the neutral density filter wheel 172 will also select 
a neutral density filter. In a step 224 the X, Y and Z 

25 offsets will be used for reconstruction of the recorded 
image at the magnification and in a step 226 the current 
position will be read from encoders in the stage which 
are accurate to .10 micron. 

In order to identify the selected region the 

30 mouse is moved to that area of the region in a pointing 
operation in a step 240 as shown in FIG. 9. The mouse 
may be moved to draw a box around the region selected. 
In a step 242 the X and Y screen points are computed for 
the edges of the regions selected and the computed image 

35 or pixel points are translated to stage coordinate points 
in order to control the stage of the microscope. In a 
step 244 a list of all of the X fields for positioning 
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the stage for the objective is stored in random access 
memory and may be backed up on the hard disk. The 
information from the X offsets for the objective and the 
stage offsets is used as well as the size of the field to 
5 position the slide properly under the objective to 
capture the micro image. 

When the slide has been positioned properly, as 
shown in FIG. 10 in a step 250 the stage is positioned 
for each of the X and Y coordinate values in stage 

10 coordinate values and the digitized image is captured by 
the cameras and stored in RAM and backed up on the hard 
disk. The image may be then analyzed quantitatively in 
various manners such as those set forth in the 
previously- identified United States application. 

15 Optionally the image may be stored for archival purposes 
in a step 254 . 

In order to override the specific control 
functions that take place as shown in FIG. 7, a screen 
is provided as shown in FIG. 8 wherein the XY step size 
can be edited, the X, Y and Z offset can be edited, the 
lamp voltage can be selected, the neutral density filter 
can be selected as well as the opening of the field 
diaphragm and several other microscopic characteristics. 
FIG. 8 is a view of the settings of the microscope 
25 objective properties of the Axioplan 2, computer- 
controlled microscope. 

The X and Y positioning is specifically carried 
out as shown in FIG. 11 where the slide 18 is shown with 
a slide boundary 270, 272, 274 and 276. Stage boundary 
for limits of the stage travel for purposes of the stage 
the stage can be moved all the way from an upper left 
hand corner of travel 276 to a lower right hand corner of 
travel 280. At the upper left hand bounded corner of 
travel 278 limits which a signal that the end of travel 
35 has been reached and the stage is then translated a short 
distance 282 in the extra action and a short distance 284 
in the Y direction to define the first tile 288 in terms 
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of a reference point 290 at its upper left hand corner. 
Since the size of the macro image tile 288 is known, the 
next macro image tile 292 may be placed contiguous with 
it by moving the stage appropriately and by measuring the 
5 location of the stage from the stage in counters without 
the necessity of performing any image manipulation. The 
image tiles 288 and 292 may be abutted without any 
substantial overlap or they may be overlapped slightly, 
such as a one pixel with overlap, which is negligible 

10 insofar as blurring of any adjacent edges of abutted 

image tiles. The upper left hand corner 300 of the tile 
292 defines the rest of 292 and other tiles can be so 
defined. Micro image tiles can likewise be defined so 
that they are contiguous but not substantially 

15 overlapping, as would interfere with the composite image. 
This avoids the problems encountered with having to 
perform extended computations on digital images in a 
frame storer or multiple frame storage in order to match 
or bring the images into contiguity without blurriness at 

20 the edges of contiguous image tiles. It may be 

appreciated as shown in FIG. 2 that the low power image 
24 has a plurality of micro images defined therein which 
are tiled and which are shown in higher magnification as 
individual tiles 312, 314, 316 and the like in FIG. 2. 

25 In addition, the region 310 when magnified as shown in 
the window 26 may exceed the bounds of the window and 
thus the window may include scroll bars or other means 
for allowing the image 310 which is larger than the 
window 26 to be examined from within the window 26. 

30 The stage 200 is best seen in FIG. IIA and . 

includes the X and Y stepper motors 279 and 281 with 
their respective encoders, which provide a closed loop 
system to give the . 1 micron accuracy versus the usual 5 
or 6 micron accuracy of most microscope stages without a 

35 closed loop system. This closed loop system and this 

very high accuracy allow the abutting of the tile images 
for both high magnification and low magnification images 
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without the substantial overlap and the time-consuming 
and expensive software currently used to eliminate the 
overlap and blurriness at the overlapping edges of 
adjacent image tiles. With the precisely positioned 
stage and by using the tiling system described in 
connection with FIG. 11, where the slide is precisely 
positioned relative to a center point CP for the slide, 
and the known position of point 278 is always taken from 
the same point, the tiles may be positioned precisely in 
a horizontal row and precisely in vertical rows to 
reconstruct the macro image and the micro image. This 
reconstruction is done without the use, as in the prior 
art, of extensive software manipulation to eliminate 
overlapping image tiles, horizontally or vertically or 
15 the haphazard orientation of image tiles. 

Furthermore, as shown in FIG. 3, the low power 
window 24, high power window 26 and control window 28 can 
be used in conjunction with reporting of quantitative 
analysis data, histograms, etc. for the specimen being 
20 viewed; and such analysis information may be provided as 
a visual output in a window 320. Each of the various 
regions 30 that a pathologist may follow in marking 
various features in the low power window 24 and the high 
power window 26 may be reflected in both windows in order 
25 that an audit trail is provided for the system. 

The present invention also includes the facility 
for allowing remote diagnostics to occur by being able to 
couple the system either over a network communication 
facility to an intranet, for instance via the network 
interface, or via a modem or other suitable connection, 
to an internet so that once the image has been scanned 
and stored in memory on hard disks or other storage, 
remote users may be able to access the low magnification 
image as well as the high magnification image and move 
35 around within both images to make determinations as to 
the histological characteristics of. the samples via Z 
scores . 
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An additional feature of the system includes a 
plurality of networked workstations coupled to a first 
computer console 12 having a display screen 22 connected 
to the microscope 14. Satellite work stations 350 and 
5 352 are substantially identical to the work station 12 
including respective computers 354 and 356 coupled to 
displays 358 and 360. The devices can be manipulated 
through input devices 360 and 362 which may include a 
keyboard, mouse and the like. Also a third device can be 

10 connected including a work station 370, having a display 
372, a computer 374 and an input device 376. Each of the 
devices is connected over respective network lines 380, 
382, 384 to the computer 12 which transmission may be via 
either net or the like. Each of the different operators 

15 at the physically separate viewing stations can locate 

regions from the view of entire tissue cross sections via 
a macro view and label the regions for subsequent 
scanning and/or quantitative analysis. A single operator 
at the instrument station 12 can locate regions to view 

20 the entire tissue cross section. Those regions can be 
labeled for subsequent scanning and/or quantitative 
analysis with subsequent review and physically remote 
viewing stations, for instance, in an operating room or 
in individual pathologists' signout areas in order to 

25 review analysis results while still maintaining and 

reviewing the entire macro view of the tissue and/or the 
individual stored images from which the quantitative 
results were obtained. The viewing stations 350, 352 and 
370 can comprise desk top computers, laptops, etc. There 

30 is no need for a microscope at the network stations 350, 
352 and 370. 

In a still further alternative embodiment, 
remote workstations 400, 402, 404, 406 and 408 may be 
connected through a server 410 which may be supplied via 

35 a packet switched network. The server 410 and may be a 
hypertext transport protocol based server of the type 
used for the World Wide Web or may be a telnet type 
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server as used previously in internet remote operation 
applications. The server 410 communicates via a 
communications channel 414 with a local computer 416 
having a display 418 associated therewith, the local 
5 computer 416 being connected to the microscope 420. Each 
of the remote work stations 400, 402, 404, 406 and 408 
may perform the same operations as the stations 350, 352 
and 370 although they do it from nearby buildings or even 
from around the world, thus providing additional 

10 flexibility for others to make use of the specimen 

obtained and being viewed under the microscope 420. In 
addition, stored images may be disseminated through the 
server 410 to the remote servers 400 through 408 for 
further analysis and review. 

15 In FIG. 14, there is illustrated on screen 28 a 

basal layer 431a of a cut cross-section of a rat 
esophagus. The basal layer is elongated and linear in a 
downward direction, and the selected point of interest is 
shown as a box 30 on the basal layer on the composite, 

20 low magnification image. The high magnification image 26 
of this selected point of interest is shown on screen 26 
in FIG. 14A. In FIG. 15 is shown a mouse colon as a 
reconstructed, low magnification macro image 28 which has 
been reduced l/l6th in size. The micro image 26 is shown 

25 in FIG. 15A, and the marking therefore is shown in FIG. 
15. 

The analysis for texture and for morphological 
features used to analyze a series of regions 30 on the 
elongated basal layer that were analyzed at high 
30 magnification are shown in FIGS. 16, 16A and 17. The 

manner of doing these tests and of obtaining a Z score or 
grade is disclosed in the aforesaid patent application. 

While there has been illustrated and described a 
particular embodiment of the present invention, it will 
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be appreciated that numerous changes and modifications 
will occur to those skilled in the art, and it is 
intended in the appended claims to cover all those 
changes and modifications which followed in the true 
5 spirit and scope of the present invention. 
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What Is Claimed Is; 



1. 



A method of using a computer- controlled 



microscope to acquire and store multiple resolution 
images from a specimen on a microscope support, said 
5 method comprising the steps of : 

digitally scanning and storing a first set of 
digitized images taken from a portion of a specimen on a 
microscope slide in a first set of contiguous image tiles 
at a first magnification to allow formation of an overall 

10 view at a first magnification resolution; 

digitally scanning and storing a second set of 
digitized images taken from the portion of the specimen 
on the microscope slide in a second set of contiguous 
image tiles at a second, higher magnification; and 

15 addressing the first and second digitized images 

so that the higher magnification images can be easily 
located from the lower magnification images. 



step further comprises providing each of the image tiles 
20 in the first series and in the second series with 

coordinate information for enabling reconstruction of the 
first and second images. 

3 . The method of Claim 1 wherein each of the 
digitized images of the series corresponds to 
25 substantially an optical view as seen by a person through 
an objective lens of the microscope. 



remotely operating the controlled microscope to select a 
segment of the specimen from the first magnification view 
30 for digitally scanning and storing the second set of 
images tiles. 



2. 



The method of Claim 1 wherein the addressing 



4. 



The method of Claim 1 including the step of 



5. A method of using a computer-controlled 
microscope imaging system for acquiring and storing 
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digitized microscopic images from a specimen on a 
microscope support, said method comprising the steps of: 

scanning and digitizing a specimen at a low 
magnification through a microscope to acquire digitized 
5 images ; 

displaying to the observer a low magnification, 

digitized image of the specimens- 
selecting a segment of the specimen from the low 

magnification digitized image for viewing at a higher 
10 magnification greater than the low magnifications- 
scanning the selected segment at the higher 

magnification and digitizing the image of the segment to 

acquire digitized images; and 

storing the respective, low magnification, 
15 digitized images and the high magnification, digitized 

images of the segment . 

6 . The method of Claim 5 wherein the step of 
scanning and digitizing at low magnification further 
includes the steps of: 

20 positioning and coordinating spatially a 

plurality of low magnification image tiles spatially to 
provide a composite macro image of the specimen formed 
from the plurality of low magnification image tiles; and 

providing a macro image of the specimens having 

25 a field of view substantially larger than the field of 
view yielded by the objective lens used for the low 
magnification scanning. 



7. A method in accordance with Claim 6 
including the step of reducing the size of the composite, 

30 macro image before displaying the composite macro image 
of the specimen to the viewer. 

8 . A method in accordance with Claim 6 wherein 
the step of selecting and digitizing the low 
magnification, digitized image further includes the step 
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of positioning and coordinating spatially a plurality of 
low magnification images spatially to provide a composite 
image having a field of view substantially larger than 
the field of view yielded by the objective lens used for 
5 high magnification scanning. 



9. A method in accordance with Claim 5 in which 
the step of the selecting a defined segment area of 
interest further includes the step of moving a marker on 
the low magnification image to a point of interest to be 

10 viewed at high magnification; and operating the 
microscope to change objectives and conditions of 
illumination to provide a good image at high 
magni f icat ion . 

10. A method in accordance with Claim 9 wherein 
15 the high and low magnification images are each 

reconstructed from image tiles each having a spatially 
known location used to assure their proper positioning in 
the reconstructed image. 



11. A method in accordance with Claim 10 
20 wherein the step of automatically changing from one 

magnification scanning operation to another magnification 
scanning operation which includes one or more of the 
steps of: 

changing the microscope objective lens to a 

25 different magnification; 

changing at least one filters- 
changing the field diaphragms- 
changing the light intensity; 
changing the focus; and 

30 shifting the microscopic stage. 



12. The method of Claim 9 including the step of 
displaying the low magnification image of the specimen 
and displaying the marker on the low magnification image, 
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and simultaneously displaying the high magnification 
image of the segment at a position located by the marker 
on the low magnification image. 

13 . The method of Claim 5 including the step of 
5 shifting the marker to different segments and viewing at 
high magnification each of the selected segments; and 
marking on the low magnification image, each of the 
segments viewed at high magnif ication. 

14. . The method of Claim 5 including the steps 
10 of locating a computer-controlled microscope system used 
to make and store the digitized images on the Internet 
having a web site therefor; including the step of 
connecting an observer's computer and viewing screen over 
the Internet to the computer-controlled microscope at the 
15 web site in real time and transmitting the digitized 
signals for low magnification image and high 
magnification image from the web site to the observer's 
computer and viewing screen. 

15. The method of Claim 5 including the step of 
20 enhancing the reconstructed, high magnification image by 

color filtering. 

16 . The method of Claim 5 including the step of 
providing a full color, low magnification, digitized 
image to the observer for aiding the observer in 

25 selecting a plurality of areas of interest on a priority 
basis . 

17. The method of Claim 5 including the steps 

of : 

analyzing the specimen on the slide with image 
30 analysis tests; and 
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reporting the results of the tests to the 
observer while at least one of the respective images is 
being viewed by the observer. 

18 . The method of claim 17 wherein the specimen 
5 is a biological specimen, and the analyzing step 

including performing analysis tests on tissue or cells in 
the biological specimen. 

19. A method in accordance with Claim 5 
including the step of the observer changing the 

10 magnification of the high magnification image to a 

magnification intermediate the high and low magnification 
images by the use of an image analysis algorithm. 

20. A method in accordance with Claim 5 
including the step of and scrolling the high 

15 magnification image tiles to bring into view on a screen 
adjacent sections not previously viewed. 

21. A method of using a computer-controlled 
microscope imaging system for acquiring and analyzing 
areas of interest, said method comprising the steps of: 

scanning and digitizing a specimen at a low 
magnification through a microscope; 

displaying to the observer a low magnification, 
digitized image of the specimen; 

selecting a segment of the specimen from the low 
magnification digitized image for viewing at a higher 
magnification greater than the low magnification; 

automatically scanning the selected segment at 
the higher magnification and digitizing the image of the 
segment; and 

making available in real time to the view of the 
observer the respective, low magnification, digitized 
image and the high magnification, digitized image of the 
segment . 
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22. The method of Claim 21 wherein the step of 
scanning and digitizing at low magnification further 
includes the steps of: 

positioning and coordinating spatially a 
5 plurality of low magnification image tiles spatially to 
provide a composite macro image of the specimen formed 
from the plurality of low magnification image tiles; and 

providing a macro image of the specimens having 
a field of view substantially larger than the field of 
10 view yielded by the objective lens used for the low 
magnification scanning. 

23. A method of using digitized, microscope 
images of a specimen having images at a first resolution 
at a low magnification and images at a second resolution 

15 at a higher magnification, the method comprising the 
steps of : 

showing to a user a series of digitized, tiled 
and stored images abutted to form. a macro image from the 
specimen at a first resolution; 
20 selecting from the stored macro image an area 

thereon to the viewer to be viewed at higher resolution; 

showing to the user a series of abutted, 
digitized, tiled, stored images at the higher 
magnification and higher resolution to the viewer to be 
25 viewed at higher resolution. 

24 . A method in accordance with Claim 23 
including the further step of shifting by user, back and 
forth between lower and higher resolution, stored images 
to analyze the specimen. 

30 25. A method in accordance with Claim 24 

including the further step of marking on the stored macro 
image the area to be viewed at higher resolution. 
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26. A method in accordance with Claim 23 
including the step of scrolling to neighboring, tiled 
images for viewing areas on the specimen adjacent the 
selected area. 

27. A method of acquiring and analyzing 
biological specimens with a computer- controlled 
microscope, said method comprising the steps of: 

scanning and digitizing a series of fields of 
views through a microscope objective lens of a low 
magnification and providing a series of digitized fields 
of view images; 

arranging and positioning the adjacent fields of 
view images and forming therefrom a composite, macro 
image in color of the specimen having a field of view 
larger than that of the objective lens; 

displaying the macro color image to the observer 
to provide the observer with a larger field of view of 
the specimen than the observer could see through an 
objective lens on the microscope; 

selecting a segment of the specimen from the 
macro image for viewing at a higher magnification than 
the low magnification; 

scanning and digitizing a series of fields of 
view through a microscope objective lens of a higher 
25 magnification and providing a series of digitized images- 

arranging and positioning the adjacent fields of 
view images at high magnification and forming therefrom a 
composite micro image of the segment having a field of 

view larger than that of the high magnification objective 
30 lens; and 

displaying to the observer both colored 
composite macro image and the micro image each having a 
field of view substantially larger than their respective 
objective lens. 
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28. A method in accordance with Claim 27 
including the step of marking on the macro image, the 
location of the micro image, so that the observer 
understands where the micro image is located on the macro 

5 image . 

29. A method in accordance with Claim 27 
including the step of selecting several segments each for 
viewing as a micro image and marking the location of each 
micro image on the composite image, 

10 30. A method in accordance with Claim 29 

including the step of recording the macro image with the 
marks thereon to provide a record of the segments viewed 
by the observer at high magnification to provide an audit 
trail for a later auditing of the segments viewed at 

15 higher resolution. 

31. A method of acquiring and analyzing 
biological specimens with a computer -control led 
microscope; said method comprising the steps of: 

providing a biological specimen in position for 
20 scanning through the computer-controlled microscope; 

scanning and digitizing the biological specimen 
through the microscope at a first, low magnification and 
acquiring and storing a digitized image of the specimen 
in color; 

25 displaying the stored, low magnification, 

digitized image in color to an observer to provide a 
macro image for viewing by the observer; 

selecting a segment of the colored, digitized, 
low magnification image for viewing at a higher 

30 magnification than the low magnification; 

scanning the segment at the higher magnification 
and acquiring a colored digitized image of the segment at 
the higher magnification to provide a micro image of the 
segment ; 
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making available in real time to the view of the 
observer, the respective micro and the macro images; and 

performing a quantitative analysis of the 
specimen from the digitized high magnification image. 

32. A method in accordance with Claim 31 
including the step of marking on the macro image the 
position of the micro image so that the viewer 
understands the location of the micro view. 

33. A method in accordance with Claim 31 
10 including the steps of : 

digitizing a plurality of fields of view through 
the microscope at each of the low and high magnifications 
to form a series of digitized image tiles; 

arranging the respective image tiles to form the 
15 respective macro and micro images; 

displaying to the observer the micro and macro 
digitized images that each have a larger field of view 
than a single field of view through respective objective 
lens for low and high magnification. 

^° 34. A microscopic system for analyzing a 

specimen and a segment of the specimen, said apparatus 
comprising: 

a microscope having a plurality of objective 
lens to acquire images at different magnifications; 

a computer-controlled light illumination 
subsystem on the microscope for adjusting light 
illumination for different magnifications; 

a computer-controlled, focusing subsystem on the 
microscope for adjusting the focus for different 
30 magnifications; 

a computer- controlled X and Y stage to position 
the specimen to be viewed at specified X and Y 
coordinates; 
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an imaging subsystem connected to a microscope 
to acquire and digitize the images of the specimen; 

a first optical system and image acquisition and 
image storing system having a first objective lens to 
5 acquire and arrange a stored series of images at a low 
magnification to provide a macro digitized image of the 
specimen having a field of view larger than the field of 
view from the objective lens used for the low 
magnification acquisition of images ; 

10 a second optical and image acquisition and image 

storing system having a second, higher magnification 
objective lens to acquire and arrange a stored series of 
images at higher magnification and to provide a micro 
view image of the segment having a field of view larger 

15 than the field of view from the higher magnification 
objective lens. 

35. A system in accordance with Claim 34 
wherein the first image subsystem includes an X and Y 
position storage device for storing the X and Y 

20 coordinates for the macro image. 

36. A system in accordance with Claim 34 
wherein the second image acquisition subsystem has a 
subsystem for reconstructing high magnification image 
tiles with their X and Y coordinates positioned to 

25 reproduce spatially high resolution image tiles into the 
micro image as if the micro image was in the original 
image . 

37. A system in accordance with Claim 34 
wherein the first image acquisition subsystem has a 

30 subsystem for reconstructing low resolution image tiles 
with their X and Y coordinates positioned to reproduce 
spatially the low resolution tiles into a macro image of 
the specimen. 
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38. A microscopic system for analyzing a 
specimen and a segment of the specimen, said apparatus 
comprising : 

a microscope having a plurality of objective 
5 lens to acquire images at different magnifications; 

a computer- control led light illumination 
subsystem on the microscope for adjusting light 
illumination for different magnifications; 

a computer-controlled, focusing subsystem on the 
10 microscope for adjusting the focus for different 
magnifications ; 

a computer-controlled X and Y stage to position 
the specimen to be viewed at specified X and Y 
coordinates; 

15 an imaging subsystem connected to a microscope 

to acquire and digitize the images of the specimen; 

a first optical system and image acquisition 
system having a first objective lens to acquire and 
arrange a series of images at a low magnification to 

20 provide a macro digitized image of the specimen having a 
field of view larger than the field of view from the 
objective lens used for the low magnification acquisition 
of images; 

a second optical and image acquisition system 
25 having a second, higher magnification objective lens to 
acquire and arrange a series of images at higher 
magnification and to provide a micro view image of the 
segment having a field of view larger than the field of 
view from the higher magnification objective lens; and 
30 a display screen device to make available in 

real time the micro and macro images on screens of the 
screen device. 



39. A system in accordance with Claim 38 
wherein the first image subsystem includes an X and Y 
35 position storage device for storing the X and Y 
coordinates for the macro image. 
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40. A method for analyzing biological specimens 
by an image analysis system having a computer- controlled, 
automated microscope comprising the following steps: 

placing a biological specimen in a microscope 
5 for viewing; 

using a computer terminal to control the image 
analysis system and microscope to acquire low 
magnification, image tiles of the specimen and to provide 
low magnification, composite, macro image from the 
10 plurality of image tiles ; 

displaying the macro image on a screen of the 
computer terminal; 

interactively selecting at least one point of 
interest on the displayed macro, specimen image for 
15 viewing at a higher magnification; 

sending signals from the computer terminal to 
control the image analysis system and the computer- 
controlled microscope to acquire a plurality of higher 
magnification, image tiles and to provide a low 
20 magnification, composite micro image; and 

analyzing the micro images of the specimen. 

41. A storage medium having digitized images of 
a specimen from a microscope slide comprising: 

a storage medium; 
25 a first series of contiguous, multiple digitized 

images at a first magnification abutted against each 
other to create an overall low resolution view of several 
adjacent, original microscope images assembled together; 

the first series of multiple, digitized images 
30 taken from at least a portion of a specimen on the slide; 

a second series of contiguous, multiple, 
digitized images at a second higher magnification abutted 
against each other to create a high resolution view of 
several adjacent, original microscope images assembled 
35 together, taken from said portion of the specimen; and 
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the first and second series of digitized images 
providing multiple resolution images of the slide 
specimen to a viewer. 



42. A storage medium in accordance with Claim 
5 41 wherein each of the series of contiguous, multiple 

images corresponds to an optical image seen by a person 
through an objective lens of a microscope. 

43. A storage medium in accordance with Claim 
41 wherein each of the series of contiguous, multiple 

10 images comprises reduced image of a corresponding 
original microscope image. 

44 . A storage medium in accordance with Claim 
41 wherein the storage medium comprises a RAM. 



45. A storage medium in accordance with Claim 
15 41 wherein the storage medium comprises a hard disk. 

46. A storage medium in accordance with Claim 
41 wherein the stored images may be disseminated through 
a server to remote devices located on a communication 
channel . 
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METHOD AND APPARATUS FOR ACQUIRING 
AND RECONSTRUCTING MAC^IFIED SPECIMEN IMAGES 
FROM A COMPUTER -CONTROLLED MICROSCOPE 



Field of the Invention 
5 This invention relates to a method of and 

apparatus for acquiring and recording digital images of 
an optical image viewed through a computer-controlled 
automated microscope and also, to using the latter in a 
quantitative analysis of plant or biological specimens. 

10 Background of the Invention 

In the image analysis and quantification of DNA 
from tissue sections as disclosed in United States Patent 
4,741,031, and also especially in the immunohisto- 
chemistry assays on the kinds of cell analysis systems 

15 disclosed in United States Patents 5,086,476; 5,202,931; 
and 5,252,487 issued to Bacus, there is a problem of 
first locating the cancer regions for analysis under low 
power and then remembering them when performing the 
analysis under higher power. There is a need, a 

20 requirement to image and digitally record an object in a 
relatively flat plane at high resolution/magnification. 
Today, it is impractical to construct an optical image 
sensor large enough to cover the entire image area e.g., 
of a specimen on a microscope slide, at the required 

25 resolution. This is because lens size and 

resolution/magnification issues limit the size of the 
field of view of magnified objects and their resulting 
images . Viewing through a microscope is akin to viewing 
through a periscope in that one sees a very small field 

30 of view even at low magnifications, such as 1-25X- A 
pathologist using a microscope often scans a slide to 
obtain in his mind an overall view or sense of what 
constitutes the specimen and he remembers the general 
locations of the diagnostically significant, small pieces 

35 of the specimen. Usually, these are the diseased areas, 
such as malignant or potentially malignant portions of 
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the specimen. To obtain higher resolution and 
magnification of these suspicious portions, the 
pathologist switches to a higher magnification objective 
lens but then the field of view becomes much smaller 
5 again. Often, the pathologist switches back and forth 
between the lower magnification, larger field of view 
objective lens to orient himself relative to the specimen 
and the high magnification, smaller field of view to 
obtain the detailed, high resolution view of the 

10 suspicious area on the specimen. Thus, the user never 
receives a magnified, condensed overall view of the 
specimen or a portion of the specimen but must remember 
the series of views taken at low magnification. 
Likewise, at high resolutions and high magnifications, 

15 the user never receives or views a collection of adjacent 
images but must interrelate these successive images in 

the user's mind. 

Stated differently, the problem is that once the 
microscope is set up for quantitation by image analysis 

20 under, e.g. 40x, where all of the diaphragms are set and 
light adjusted, etc., if the operator needs to move to 
another tissue area, it is first desirable to locate it 
at e.g. lOx. In fact, often regions can only.be located 
at this power. In order to do so however, all of the 

25 settings (diaphragms, light levels, wavelengths of light, 
etc.) have to be changed to view the tissue at this 
magnification. Currently, there is no way to ensure that 
one could go back to the settings at the previous 40x 
magnification and continue on with the quantitative image 

30 analysis of that same specimen. This necessitates 
finding those areas under 4 Ox, without changing 
objectives, which is very slow and time-consuming, and 
often important cancer areas can be missed. 

Also, another problem with tissue analysis, at 

35 its current state-of-the-art, is that it is not 

completely automated, for example, with regard to finding 
structural regions such as glands, basal layers or other 
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important diagnostic regions. However, as set forth in 
my co-pending patent application Serial No. 701.974. 
filed August 23, 1996, if these regions are located, 
important very sensitive diagnostic measurements can be 
performed, which patent application is incorporated by 
reference as if fully reproduced herein. For example, as 
disclosed in the aforesaid patent application, assays are 
made of a variety of tissue types, both human and animal 
for analysis of neoplasia in tissue, for pre-invasive 
cancer in tissue, and the effects on the tissue of 
chemopreventive agents. A quantitative analysis by image 
processing techniques is performed on tissue types, 
having various architectural features, such as breast 
tissue, colon tissue, prostate tissue, esophageal tissue. 
15 skin tissue, cervix tissue, etc. These tissues have 
different morphologies, and they undergo different 
neoplasias usually resulting from a cellular mutation, as 
may be enhanced by a carcinogen, or resulting from a 
cellular proliferation rate enhanced by hormones, growth 
20 factors, or other inducers of abnormal tissue growth. 
Often it is desired to quantify small changes in the 
neoplasia when it is incipient or through a series of 
analyses performed at close time intervals to measure 
whether the neoplasia progression is increasing or has 
25 been slowed, stopped or regressed. 

Usually, the tissue specimens are cut to expose 
the basal layer for review under the microscope. 
Typically, the quantitative measurements are performed at 
40x to obtain 100 to 400 tissue images. The 40x 
objective provides a narrow field of view of a very small 
portion of the entire basal layer. Often, the basal 
layer is somewhat elongated and generally linear such as 
a basal layer in a rat esophagus; and the analysis of the 
basal layer requires examining it along its length. The 
35 basal layer in a mouse colon is more in the form of an 
irregular, circular shape; and the analysis of this basal 
layer requires traveling about this circular shape. In 
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breast tissue samples, suspected tumor areas may be at 
widely-spaced points in the stained tissue; and one wants 
to be able to navigate and travel to these specific 
suspected areas and to do the 4 Ox analysis at these areas 
5 in an efficient manner. There is a need to allow an 
experienced operator to interact with the analysis to 
locate and identify such regions in an interactive 
manner. Especially, an interactive manner that would be 
familiar and consistent with the use of a microscope 

10 manually, with high power magnification and low power 

magnification simultaneously available, but performed on 
a computer terminal. Such a level of interaction is 
different than the interaction with the system disclosed 
in the above-listed Bacus Patents. There is a need to 

15 take the level of interaction to a higher level and let 
each component, the human and the computer, perform the 
part that it does best, in the most cost-effective 
manner. 

The current state of archiving the digital 

20 images achieved through a microscope is often by having 
photographs or by video tapes. The photographs are 
difficult to use as is a video tape particularly when the 
user wants to move rapidly back and forth between various 
images and to scroll through various adjacent parts of 

25 the specimen image. Further, current archival methods 

lack an overall macro image of the specimen, which allows 
the user to know exactly where the particular high 
resolution view is from when the user is making an 
analysis of the high resolution image. 

30 While digitized images can be stored 

magnetically or otherwise digitized and recorded on 
various recording mediums, no current archival system 
allows the user to toggle back and forth between high 
magnification images and low magnification images or 

35 between various images at different magnifications such 
as that achieved by a pathologist switching microscope 
objective lenses in real time to get the macro and micro 
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images from the same location on the specimen. 
Heretofore, the practice of pathology has been relatively 
limited to the use of microscopes and to the pathologist 
having to use the microscope to review the particular 
5 specimen. 

There is a need for a dynamic system whereby one 
or more or several pathologists, including a consulting 
pathologist, may view the same area simultaneously and 
interact with one another either in diagnosis or in 

10 analysis. Also, it would be best if the images from the 
specimen could be stored so that a pathologist could 
easily examine the images at his leisure using an 
intranet or Internet browser at a later date merely by 
accessing the particular web site where the images are 

15 located. 

A similar problem exists on the Internet or 
intranet where a pathologist may receive a single field 
of view magnified image taken from a specimen over the 
Internet or the intranet on his browser. The pathologist 
20 must be provided with explanations to coordinate the high 
resolution view with the lower resolution view. The 
number of views available to the pathologist is very 
limited, and the pathologist is unable to select other 
views or to scroll to neighboring views at the areas that 
25 are most interesting to the pathologist. 

There are available on the market computer- 
controlled, automated microscopes such as those sold by 
Carl Zeiss, Inc., Thornwood, N.J., under the name 
Axioplan 2 for taking photographic images of a specimen 
30 in the microscopic field of view. Those particular 
microscopes have computer-controlled and automatically 
adjusted subsystems, such as an illumination subsystem, a 
focusing siibsystem, a diaphragm or optical stops 
subsystem, an objective lens subsystem, or a filtering 
35 subsystem. As an operator selects changes from one 

objective lens, such as one providing low magnification, 
e.g., 4x, to a higher magnification, e.g., 40x, the 
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computer-automated system will turn the lens turret to 
switch in the high magnification automatically and 
adjusts the lens and also automatically adjusts the 
illumination to eliminate glare and to provide the proper 

. 5 light illumination including light density. Further, the 
focus is adjusted, and the proper diaphragm openings are 
automatically reset. Thus, the computer-controlled, 
automated subsystems automatically reset to values stored 
and predetermined for each selected objective lens and 

10 the analysis being done. 

Those particular microscopes can be used to view 
various objects or specimens, but are most typically used 
to view and to take still photographs of biological 
specimens, such as tissues and cells. Those particular 

15 microscopes lack a computer-controlled X and Y stage for 
translating a specimen -carrying slide with respect to the 
field of view of the selected objective lens. Currently, 
pathologists and others who use such microscopes want to 
view the specimen images in full color or in enhanced 

20 colors using fluorescent illumination and/or 
monochromatic images using the automated filter 
subsystems on the microscopes. Currently trained 
pathologists or clinicians are accustomed to manually 
adjust and have a microscope available to them to view 

25 larger areas of the specimen at low magnification, and 
then to momentarily switch in a new higher magnification 
lens to obtain a more highly magnified image of a portion 
of the specimen viewed at low magnification. 
Pathologists and those working in this area have created 

30 in themselves a desire to view suspect tissue as though 
they were viewing the specimen tissue through a 
microscope and appear to resist analysis systems that do 
not provide them this ability. 

The microscopic field of view reduces very 

35 substantially as the magnification increases. The skill 
level of the clinician and/or pathologist is important to 
locate viewing the most suspicious areas or points of 
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interest on the specimen. Sometimes, a technician will 
do a first assay and analysis. A pathologist will return 
to the selected points of interest or other points of 
interest for review and analysis . One concern with 
5 respect to a quantitative analysis of breast cancer 

tissue or prostate biopsy tissue samples to pap smears or 
other tests for various cancers or the like is that a 
particularly suspicious point in the tissue may be 
overlooked and missed during the visual assay or for 

10 selection for an automated review analysis. When 

observing at high magnifications', the field of view is 
limited to very small area of the specimen. Hence, the 
observer has difficulty in knowing and remembering the 
actual, exact location of this small periscopic view 

15 within the very large whole specimen. 

Often, also the problem is finding or locating 
the tissue or cells for view at high magnification so 
that artifacts and/or blank spaces on the slide are not 
viewed. A number of approaches have been proposed to 

20 prescreen and locate by an X and Y address the cells or 
small points of interest from a very large number of 
potential points of interest. 

There are currently available commercial 
services for prescreening pap smears where one can mail 

25 in slides and the service will do a microscopic prescan 
at high magnification for suspected or suspicious areas 
of interest which are marked and given address locations, 
and also a video tape of the slide specimen is returned 
by this service to the sender. The sender then reviews 

30 the areas of interest located during the prescreening 
and/or the video tape to complete the analysis. 

In an attempt to locate and allow review of 
specified points of interest, U.S. Patent No. 5,428,690 
to Bacus discloses a system for prescreening of a field 

35 of cells on a specimen slide at low magnification before 
the viewer. When seeing a point of interest to be viewed 
at high magnification, the viewer will operate a switch 
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or the like to select and record the address of these 
selected prescreened points of interest. Later, these 
prescreened points of interest are then brought into 
position to be analyzed at high magnification. This 
5 system is fairly or too slow for many uses. 

A very expensive system is currently in use in 
which a pathologist located at a diagnostic center is 
able to make a diagnostic opinion with respect to 
specimens under a microscope at a remote center. The 

10 pathologist at the diagnostic center manipulates robotic 
controls to send telepathy signals via a special, 
dedicated satellite or other large bandwidth channel to 
control the microscope at the remote site in 
approximately real time. The pathologist then can 

15 maneuver the remote microscope to shift the microscope's 
field of view and to send, by telepathy, a highly 
magnified, very small image back to the pathologist. 
This system requires each subscriber to have a special 
microscope operable by manipulation of the robotic 

20 controls at the diagnostic center and a dedicated or 

large bandwidth channel to convey real time video signals 
and hence results in a very high cost for the assay being 
done. To assist the pathologist in staying within the 
specimen at the remote site, a peripheral edge or map of 

25 the specimen is made using a second video camera and a 
light box or by using computerized scanning equipment to 
trace the outline or peripheral edge of the specimen 
boundaries . A small circle of light is displayed within 
the map of the specimen so that the pathologist at the 

30 diagnostic center knows the location of the field of view 
of the highly magnified image within the specimen. In a 
sense, the pathologist is operating in real time in the 
same way that he would use his own microscope at his 
diagnostic center except for a slight transmission delay 

35 used to transmit the video signals of the highly 

magnified image over large bandwidth channel. Although 
the pathologist has a small map or peripheral outline of 
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the specimen, the pathologist's field of view of the 
actual specimen is only the small circle of view that is 
coming through the microscope objective lens. This does 
not help the pathologist locate suspicious areas of 
5 interest as in a prescreening of the entire tissue. The 
pathologist may switch to the lowest magnification to get 
the largest field of view of a small section of the 
specimen, but he never views the entire specimen at any 
magnification. Also, there is no image analysis 

10 quantitative testing from the received images at the 

diagnostic center; and no quantitative assaying is done 
with these images at the diagnostic center. 

There is a particular interest today in using 
the Internet system because it is so readily accessible 

15 by users at a low cost and using a computer and viewing 
screen connected to the computer. One problem with 
trying to do any transmission of digitized, microscopic, 
highly magnified images over the Internet is that the 
bandwidth is too narrow to accommodate the tremendous 

20 amount of stored data which needs to be transmitted. 
There is a need for a system which would allow a 
pathologist or another person, to be able to perform 
tissue analysis or quantitative assays using a standard 
computer terminal from a location remote from the 

25 automated microscope system that digitized the optical 
images . 



Summary of the Invention 

In accordance with the present invention, a 
person such as a pathologist at a computer terminal may 

30 view on a computer screen or monitor digitized images of 
a microscope specimen at different magnifications as 
selected by the person. Further, the person may receive 
on the screen a low magnification, reconstructed image of 
the entire specimen to aid the person in interactively 

35 selecting points of interest on the specimen, such as 
along a basal layer of a tissue specimen. 
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More specifically, and in accordance with the 
present invention, the microscope's small field of view 
limitation of a specimen is overcome by providing to the 
viewer a reconstructed, digitized and magnified image of 
5 the entire specimen (or a very large portion of the 

specimen) for performing a visual analysis of the entire 
tissue in full color to aid in the selection of points of 
interest to be viewed at a higher magnification and 
higher resolution. This is achieved by acquiring and 

10 storing a large number of low magnification images of the 
specimen through a microscopic scanning system, e.g., 35 
image tiles of the specimen at 1.25X, and then assembling 
and coordinating these stored tiled images to form an 
overall, low magnified image of the specimen, i.e., a 

15 macro image of the specimen. Preferably, the digitized, 
stored macro image is reduced in size by a software 
system to a smaller size, e.g., a 1/4 size image and it 
is this reduced in size macro tiled image that is 
displayed on a local screen or is sent over a low band 

20 width or a high band width channel to a remote screen. 
The pathologist not only does not need to have others do 
a slow laborious prescreening to locate suspicious areas 
for analysis or for viewing at high magnification, he can 
use his own experiences to enable him to go directly to 

25 the most suspicious areas which he sees on the macro 
image. He can, on a priority basis, do the most 
suspicious area first, followed by lower priority areas 
of interest . 

In accordance with the present invention, there 
3 0 is provided a new and improved microscopic display system 
that displays the low magnification composite image of 
the specimen to allow the user to view and to 
interactively select points of interest, each of which 
may be displayed at high magnification and resolution. 
35 This is achieved by providing the user with a marker, 

such as a cursor or the like, to select the defined area 
of interest from the stored, macro image; and to acquire 
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reproduced, spatially adjacent high resolution, digitized 
images of the selected area of interest. More 
specifically, the specimen, when it was first scanned at 
low magnification to provide a macro view of the 
5 specimen, the addresses or locations of the tile images 
and/or pixels for the composite image were acquired. 
Therefore, any selected region of interest in the macro 
image has locations to which the microscopic stage may be 
automatically repositioned under a changed, higher 
10 magnification lens to acquire higher magnification, 

digitized image tiles that can be assembled into a micro 
image. Herein, both the macro and micro images are 
formed of adjacent digitized image tiles which have been 
coordinated to reproduce spatially the original image 

15 that was scanned - 

It is the high magnification images, usually at 
40x, that were analyzed using image processing techniques 
as disclosed in the aforesaid patent application, to 
provide an assay or numerical histological data for the 

20 specimen. 

In accordance with the preferred embodiment of 
the invention, the pathologist may desire to see a larger 
region for analysis at high resolution than can be 
accommodated at this magnification on his high 

25 magnification viewing screen. The pathologist can 

quickly change the view on his screen to view adjacent, 
highly magnified, digitized image tiles at this high 
resolution by scrolling up or down or right to left to 
shift these digitized, adjacent image tiles into view on 

30 the screen. Thus, even at a higher magnification of a 
region the pathologist is able to obtain a much larger 
view than the small field for the objective lens in use 
of adjacent tissue or cells to give him a broader, 
overall perspective of what is happening or what has 

35 happened in specific section of a specimen. For 
instance, a pathologist may want to see at high 
magnification and to assay at this high magnification, a 
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suspicious area, the pathologist can draw a mark about 
the area and cause it to then be assayed and displayed. 

By having displayed on a low magnification 
screen or split screen of the full composite area, by 
5 having the high magnification region being marked on the 
low image screen, and by having the region being scrolled 
to view adjacent high magnification images on the high 
magnification screen, the pathologist has available 
information to guide him in the sense of helping to 

10 navigate intelligently within the specimen in his search 
for cancerous tissue or the like for further inspection 
or measurement of malignant characteristics. Often, when 
restricted to a field of view of an objective microscope, 
the pathologist has a difficult time, in the words of a 

15 cliche, of seeing the forest for the trees. In the 

present invention, the pathologist has a full, magnified, 
reduced in size specimen view with the higher image area 
marked thereon to guide him and to let him see the 
forest . J He can also see a region of the forest at higher 

20 magnification by scrolling adjacent tree images onto the 
high magnification screen. 

In accordance with a further aspect of the 
. invention, the user may elect to view an intermediate 
magnification and resolution. In the preferred system, 

25 at least three magnification images are acquired and 

stored as digitized images. The intermediate resolution 
images may be acquired by either switching automatically 
to a new objective lens and acquiring new digitized image 
tiles at the intermediate magnification or by using 

30 software to reconstruct a new intermediate digitized 
image from the existing high and low magnification, 
digitized images. 

The preferred, low magnification image, which is 
reduced in size, can be transmitted over narrow band 

35 width channels such as a local area network or over the 
Internet through various servers and computers . 
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Likewise, the reconstructed, high magnification images 
can be transmitted over such narrow band width channels. 

The preferred microscope is fully computer 
controlled, a pathologist or other person having a split 
5 screen computer such as a PC, can be connected to the 
microscope and operate it from a remote location with the 
computer microscope system to digitize and construct the 
macro image. Likewise, the pathologist can navigate to 
points of interest and with the computer microscope 
10 system to digitize and construct the desired micro 

images. With the present invention, there is no need for 
a specialized microscope at each remote location nor for 
a broad band channel to send video signals in real time 
between the diagnostic center and the remote location. 

15 Brief Degerlotiop of the Drawinqa 

FIG. 1 is a screen view of a system embodying 
the present invention showing a low magnification image 
of a specimen on a microscope slide in one window, a high 
magnification image of a portion of the low magnification 
20 image selected by a region marker and a control window; 

FIG. 2 is a view of a display screen of the 
apparatus embodying the present invention showing the 
control window a low magnification window having a 
plurality of high magnification micro image regions 
25 delineated therein and a high magnification window 
including one or more of the micro image regions; 

FIG. 3 is a view similar to FIG. 2 including the 
control window but also including a low magnification 
region from the slide showing regions marked by a 
30 histology grade or structure through automatic analysis 
of tissue and a high magnification window showing 
markings related to the grading or histology grade 
yielded by the automatic analysis of tissue in 
combination with a window showing a numerical score; 
35 FIG. 4 is a block diagram of the apparatus 

embodying the present invention; 
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FIG. 5 is a block diagram of a portion of the 
apparatus shovm in FIG. .4 showing details of a mechanical 
arrangement of a microscope; 

FIG. 6 is a flow diagram related to operation of 
5 the apparatus; 

FIG. 7 is a flow diagram of details of one of 
the steps in FIG. 6; 

FIG. 8 is a display screen showing control 
parameters to be manipulated thereon; 

FIG. 9 is a flow chart for a region outlying 

routine; 

FIG. 10 is a flow chart for a scanning and 
analyzing routine; 

FIG. 11 is a schematic showing of the limits of 
15 travel of the microscope stage with respect to the image 
tiles; 

FIG. IIA is a perspective view of the microscope 
stage and stepper motors and encoders providing a closed 
loop drive for the motors; 

P^G- 12 is a block diagram of a networked system 
allowing multiple workstations to obtain access to 
diagnostic image information and to manipulate such 
information locally at each workstation; 

FIG. 12A is a view of the system described in 
25 connection with FIG, 5; 

FIG. 13 is a block diagram of a remote networked 
system for distributing and accessing diagnostic images 
and data through a hypertext transport protocol based 
server directly or over a packet network; 

FIG. 14 is a view of a low magnification, 
reconstructed image from a basal layer of rat esophagus; 

FIG. 14A is a view of a high magnification, 
reconstructed image from a selected point of interest 
from FIG. 14; 
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FIG. 15 is a view of a low magnification image 
of a mouse colon having a basal layer; 
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FIG. 15A is a view of a reconstructed macro 
image of a mouse colon; 

FIG. 16 is a schematic view of an analysis from 
regions of a basal layer; 
5 FIG. 16A is a schematic view of an analysis to 

provide to a Z score; and 

FIG. 17 is a schematic view showing texture 
analysis tests for regions. 



Detailed Description of the Preferred Embodiment 

10 Referring now to the drawings, and especially to 

Figs. 4 and 5, apparatus for synthesizing low 
magnification and high magnification microscopic images 
is shown therein and generally identified by reference 
numeral 10. The system includes a computer 12 which is a 

15 dual Pentium Pro personal computer in combination with a 
Hitachi HV-C20 video camera 14 associated with a Zeiss 
Axioplan 2 microscope 16. The computer system 12 is able 
to receive signals from the camera 14 which captures 
light from the microscope 16 having a microscope slide 18 

20 positioned on an LUDL encoded motorized stage 20. The 
encoded motorized stage 20 includes a MAC 2000 stage 
controller for controlling the stage in response to the 
computer 12. A microscope slide 18 includes a biological 
specimen 21 which is to be viewed by the microscope and 

25 whose image is to be digitized both at low magnification 
and at high magnification as selected by a user. The low 
magnification digitized image is then displayed on a 21 
inch liyama video display monitor 22 having resolution of 
1600 by 1200 to provide display screens of the type shown 

30 in Figs. 1 through 3 including a low magnification image 
24, for instance, at 1.25 power, a high magnification 
image 26, for instance at 40 power and a control window 
or image 28. The low magnification image may have 
identified therein a region 30 which is reproduced at 

35 high magnification in high magnification screen or window 
26 so that a pathologist or other operator of the system 
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can review architectural regions of interest in low 
magnification image 24 and simultaneously view them in 
high magnification in the high magnification screen or 
window 26 to determine whether the cells forming a 
5 portion of the architectural feature need be examined 
further for cancer or the like or not. 

The computer 10 is constructed around a PCI 
system bus 40 and has a first Pentium Pro microprocessor 
42 and a second pentium pro microprocessor 44 connected 
10 thereto. The system bus 40 has connected to it a PCI bus 
50 and an ISA bus 52. The PCI bus 50 has a SCSI 
controller 60 connected thereto to send and receive 
information from a hard disk 62, The hard disk 62 also 
is coupled in daisy chain SCSI fashion to a high capacity 
15 removal disk and to a CD Rom drive 66. The hard disks 62 
contains the programs for operating the system for 
controlling the microscope 16 and for processing the 
images as well as for doing a quantitative analysis of 
the selected portions of the histological specimens being 
20 viewed on the slide 18. The system bus 40 also has 

connected to it a random access memory 70 within which 
portions of the program being executed are stored as well 
as a read only memory 72 for holding a bootstrap loader 
as well as portions of the basic input/output operating 
25 system. A floppy disk controller 74 is coupled to the 

system bus 40 and has connected to it a floppy disk drive 
76 for reading and writing information to a floppy disk 
as appropriate. A mouse controller 80 is coupled to the 
system bus and has a mouse 82 which operates as a 
30 pointing device for controlling manipulations on the 
screen 22 and within the windows 24, 26 and 28. A 
keyboard controller 90 is coimected to the system bus and 
has a keyboard 92 connected thereto. The keyboard 92 may 
be used to send and receive alpha numeric signals to 
35 other portions of the computer. An audio controller 100 
has a plurality of speakers 102 and a microphone 104 
connected thereto for audio input and output and is 



wo 98/44446 PCT/US98/04208 

-17- 

coupled to the system bus 40. A network interface, such 
as a network interface card 104, is connected to the 
system bus and can provide signals via a channel 106 to 
other portions of a network or internet to which the 
5 system may be connected. Likewise, signals can be sent 
out of the system through a modem 110 connected to the 
ISA bus 52 and may be sent via a channel 112, for 
instance, to the internet. A printer 116 is connected 
via a parallel I/O controller 118 to the system bus in 

10 order to provide printouts as appropriate of screens and 
other information as it is generated. A serial I/O 
controller 122 is connected to the system bus and has 
connected to it a camera controller 124 which is coupled 
to CCD sensors 126 in the cameras. The CCD sensors 126 

15 supply pixel or image signals representative of what is 
found on the slide 18 to an Epix pixci image acquisition 
controller 130 coupled to the PCI bus 50. 

The microscope 16 includes a base 140 having a 
stage 20 positioned thereon as well as an objective 

20 turret 142 having a plurality of objectives 144, 146 and 
148 thereon. The objective 144, for instance, may be of 
1.25X objective. The objective 146 may be a 20X 
objective. The objective 148 may be a 4 OX objective. 
Signals from the camera sensors and controller are 

25 supplied over a bus 128 to the image acquisition system 
where they are digitized and supplied to the PCI bus for 
storage in RAM or for backing storage on the hard disk 
62. 

When a specimen is on the slide 18 the stage 20 
30 may be manipulated under the control of the computer 

through a stage controller 160 coupled to the serial I/O 
controller 122. Likewise, a microscope controller 162 
controls aspects of the microscope such as the 
illumination, the color temperature or spectral output of 
35 a lamp 168 and the like. For instance, in normal 
operation, when a specimen is placed on the slide, 
specimen slide 18 is placed on the stage 20 in a step 
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200, as shown in FIG. 6, the processors 42 or 44 send a 
command through the system bus to cause the serial I/O 
controller 122 to signal the microscope controller to 
change magnification to 1.25X in a step 202, This is 
5 done by rotating the objective turret of the Axioplan 2 
microscope to select the objective 144. Likewise, the 
controller sets the color temperature of the lamp 168, 
sets a pair of neutral density filter wheels 170 and 172 
and sets a field diaphragm 174 for the correct 
10 illumination. A condenser diaphragm 176 is also 

controlled and a color filter wheel 180 may also be 
controlled to apply the appropriate filter color to the 
CCD censors 126 in the camera. The entire slide is then 
scanned in a step 204. The images are tiled and melded 
15 together into the overall image 24 supplied on the screen 
22 to provide the operator in the step 206 with a 
visually inspectable macro image of relevant regions of 
the slide of interest. 

In order to provide the magnified image, the 
20 mouse may be moved to identify a marker segment or region 
which, for instance, may be a rectangular region (as 
shown as 30 in FIG. 1) which will cause the microscope to 
change magnification as at step 208 to 4x, 20x, 4 Ox, 
etc., by rotating the turret to bring the appropriate 
25 objective lens system into viewing position. 

Next the user, in a step 209a, uses the mouse to 
select the region on the macro image in order to select 
the micro image to be viewed on the screen 22. In a step 
209b a test is made to determine whether the user has 
30 commanded continued inspection. If the user has, a test 
is made in a step 209c to determine if the magnification 
is to be changed by changing the selected objective. In 
the event the magnification is to be changed control is 
transferred to the step 208. If the magnification is to 
35 remain unchanged control is transferred to the step 209a. 
In the event inspection is not to continue the region 
selected is outlined for higher magnification scan in a 
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step 209d. In a step 209e, a command may be received to 
scan or acquire the higher magnification image for 
display in screen 26. The image may then be archived for 
later analysis, displayed or analyzed immediately. 
5 In order to perform the magnification called for 

in step 208, the overall illumination and control of the 
microscope will be controlled so that in a step 210 the 
objective turret 142 will be rotated to place the higher 
power objective cJ:>ove the slide 18, In a step 212 

10 voltage to the lamp will be changed to adjust the lamp 
168 to provide the proper illumination and color 
temperature as predetermined for the selected objective. 
In a step 214, the condenser diaphragm 176 will have its 
opening selected as appropriate to provide the proper 

15 illumination for that objective. In a step 216, the 

filter turret 180 will select the proper light wavelength 
filter to be supplied to the camera sensors. For 
instance, a red, blue or green filter, as appropriate, 
particularly if the specimen has been stained. In a step 

20 218 the field diaphragm 174 will have its opening 

changed. In a step 220 the neutral density filter wheel 
170 will select a neutral density filter and in a step 
222 the neutral density filter wheel 172 will also select 
a neutral density filter. In a step 224 the X, Y and Z 

25 offsets will be used for reconstruction of the- recorded 
image at the magnification and in a step 226 the current 
position will be read from encoders in the stage which 
are accurate to .10 micron. 

In order to identify the selected region the 

30 mouse is moved to that area of the region in a pointing 
operation in a step 240 as shown in FIG. 9. The mouse 
may be moved to draw a box around the region selected. 
In a step 242 the X and Y screen points are computed for 
the edges of the regions selected and the computed image 

35 or pixel points are translated to stage coordinate points 
in order to control the stage of the microscope. In a 
step 244 a list of all of the X fields for positioning 
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the stage for the objective is stored in random access 
memory and may be backed up on the hard disk. The 
information from the X offsets for the objective and the 
stage offsets is used as well as the size of the field to 
5 position the slide properly under the objective to 
capture the micro image. 

When the slide has been positioned properly, as 
shown in FIG. 10 in a step 250 the stage is positioned 
for each of the X and Y coordinate values in stage 

10 coordinate values and the digitized image is captured by 
the cameras and stored in RAM and backed up on the hard 
disk. The image may be then analyzed quantitatively in 
various manners such as those set forth in the 
previously- identified United States application. 

15 Optionally the image may be stored for archival purposes 
in a step 254. 

In order to override the specific control 
functions that take place as shown in FIG. 1 , a screen 
is provided as shown in FIG. 8 wherein the XY step size 

20 can be edited, the X, Y and Z offset can be edited, the 
lamp voltage can be selected, the neutral density filter 
can be selected as well as the opening of the field 
diaphragm and several other microscopic characteristics. 
FIG. 8 is a view of the settings of the microscope 

25 objective properties of the Axioplan 2, computer- 
controlled microscope. 

The X and Y positioning is specifically carried 
out as shown in FIG. 11 where the slide 18 is shown with 
a slide boundary 270, 272, 274 and 276. Stage boundary 

30 for limits of the stage travel for purposes of the stage 
the stage can be moved all the way from an upper left 
hand corner of travel 276 to a lower right hand corner of 
travel 280. At the upper left hand bounded corner of 
travel 278 limits which a signal that the end of travel 

35 has been reached and the stage is then translated a short 
distance 282 in the extra action and a short distance 284 
in the Y direction to define the first tile 288 in terms 
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of a reference point 290 at its upper left hand corner. 
Since the size of the macro image tile 288 is known, the 
next macro image tile 292 may be placed contiguous with 
it by moving the stage appropriately and by measuring the 
5 location of the stage from the stage in counters without 
the necessity of performing any image manipulation. The 
image tiles 288 and 292 may be abutted without any 
substantial overlap or they may be overlapped slightly, 
such as a one pixel with overlap, which is negligible 
10 insofar as blurring of any adjacent edges of abutted 

image tiles. The upper left hand corner 300 of the tile 
292 defines the rest of 292 and other tiles can be so 
defined. Micro image tiles can likewise be defined so 
that they are contiguous but not substantially 
15 overlapping, as would interfere with the composite image. 
This avoids the problems encountered with having to 
perform extended computations on digital images in a 
frame storer or multiple frame storage in order to match 
or bring the images into contiguity without blurriness at 
20 the edges of contiguous image tiles. It may be 

appreciated as shown in FIG. 2 that the low power image 
24 has a plurality of micro images defined therein which 
are tiled and which are shown in higher magnification as 
individual tiles 312, 314, 316 and the like in FIG. 2. 
25 In addition, the region 310 when magnified as shown in 
the window 26 may exceed the bounds of the window and 
thus the window may include scroll bars or other means 
for allowing the image 310 which is larger than the 
window 26 to be examined from within the window 26. 
30 The stage 200 is best seen in FIG. IIA and 

includes the X and Y stepper motors 279 and 281 with 
their respective encoders, which provide a closed loop 
system to give the .1 micron accuracy versus the usual 5 
or 6 micron accuracy of most microscope stages without a 
35 closed loop system. This closed loop system and this 

very high accuracy allow the abutting of the tile images 
for both high magnification and low magnification images 
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without the substantial overlap and the time-consuming 
and expensive software currently used to eliminate the 
overlap and blurriness at the overlapping edges of 
adjacent image tiles. With the precisely positioned 
5 stage and by using the tiling system described in 

connection with FIG. 11, where the slide is precisely 
positioned relative to a center point CP for the slide, 
and the known position of point 278 is always taken from 
the same point, the tiles may be positioned precisely in 
10 a horizontal row and precisely in vertical rows to 

reconstruct the macro image and the micro image. This 
reconstruction is done without the use, as in the prior- 
art, of extensive software manipulation to eliminate 
overlapping image tiles, horizontally or vertically or 
15 the haphazard orientation of image tiles. 

Furthermore, as shown in FIG. 3, the low power 
window 24, high power window 26 and control window 28 can 
be used in conjunction with reporting of quantitative 
analysis data, histograms, etc. for the specimen being 
20 viewed; and such analysis information may be provided as 
a visual output in a window 320. Each of the various 
regions 30 that a pathologist may follow in marking 
various features in the low power window 24 and the high 
power window 26 may be reflected in both windows in order 
25 that an audit trail is provided for the system. 

The present invention also includes the facility 
for allowing remote diagnostics to occur by being able to 
couple the system either over a network communication 
facility to an intranet, for instance via the network 
30 interface, or via a modem or other suitable connection, 
to an internet so that once the image has been scanned 
and stored in memory on hard disks or other storage, 
remote users may be able to access the low magnification 
image as well as the high magnification image and move 
35 around within both images to make determinations as to 
the histological characteristics of the samples via Z 
scores. 
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An additional feature of the system includes a 
plurality of networked workstations coupled to a first 
computer console 12 having a display screen 22 connected 
to the microscope 14. Satellite work stations 350 and 
5 352 are substantially identical to the work station 12 
including respective computers 354 and 356 coupled to 
displays 358 and 360. The devices can be manipulated 
through input devices 360 and 362 which may include a 
keyboard, mouse and the like. Also a third device can be 

10 connected including a work station 370, having a display 
372, a computer 374 and an input device 376. Each of the 
devices is connected over respective network lines 380, 
382, 384 to the computer 12 which transmission may be via 
either net or the like. Each of the different operators 

15 at the physically separate viewing stations can locate 

regions from the view of entire tissue cross sections via 
a macro view and label the regions for subsequent 
scanning and/or quantitative analysis. A single operator 
at the instrument station 12 can locate regions to view 

20 the entire tissue cross section. Those regions can be 
labeled for subsequent scanning and/or quantitative 
analysis with subsequent review and physically remote 
viewing stations / for instance, in an operating room or 
in individual pathologists' signout areas in order to 

25 review analysis results while still maintaining and 

reviewing the entire macro view of the tissue and/or the 
individual stored images from which the quantitative 
results were obtained. The viewing stations 350, 352 and 
370 can comprise desk top computers, laptops, etc. There 

30 is no need for a microscope at the network stations 350, 
352 and 370. 

In a still further alternative embodiment, 
remote workstations 400, 402, 404, 406 and 408 may be 
connected through a server 410 which may be supplied via 

35 a packet switched network. The server 410 and may be a 
hypertext transport protocol based server of the type 
used for the World Wide Web or may be a telnet type 
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server as used previously in internet remote operation 
applications. The server 410 communicates via a 
communications channel 414 with a local computer 416 
having a display 418 associated therewith, the local 
computer 416 being connected to the microscope 420. Each 
of the remote work stations 400, 402, 404, 406 and 408 
may perform the same operations as the stations 350, 352 
and 370 although they do it from nearby buildings or even 
from around the world, thus providing additional 
flexibility for others to make use of the specimen 
obtained and being viewed under the microscope 420. In 
addition, stored images may be disseminated through the 
server 410 to the remote servers 400 through 408 for 
further analysis and review. 

In FIG. 14, there is illustrated on screen 28 a 
basal layer 431a of a cut cross-section of a rat 
esophagus. The basal layer is elongated and linear in a 
downward direction, and the selected point of interest is 
shown as a box 30 on the basal layer on the composite, 
low magnification image. The high magnification image 26 
of this selected point of interest is shown on screen 26 
in FIG. 14A. In FIG. 15 is shown a mouse colon as a 
reconstructed, low magnification macro image 28 which has 
been reduced 1/I6th in size. The micro image 26 is shown 
in FIG. 15A, and the marking therefore is shown in FIG. 
15. 
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The analysis for texture and for morphological 
features used to analyze a series of regions 30 on the 
elongated basal layer that were analyzed at high 
magnification are shown in FIGS. 16, 16A and 17. The 
manner of doing these tests and of obtaining a 2 score or 
grade is disclosed in the aforesaid patent application. 

While there has been illustrated and described a 
particular embodiment of the present invention, it will 
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be appreciated that numerous changes and modifications 
will occur to those skilled in the art, and it is 
intended in the appended claims to cover all those, 
changes and modifications which followed in the true 
5 spirit and scope of the present invention. 
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What Is Claimed la : 

1. A method of using a computer-controlled 
microscope to acquire and store multiple resolution 
images from a specimen on a microscope support, said 
5 method comprising the steps of : 

digitally scanning and storing a first set of 
digitized images taken from a portion of a specimen on a 
microscope slide in a first set of contiguous image tiles 
at a first magnification to allow formation of an overall 
10 view at a first magnification resolution; 

digitally scanning and storing a second set of 
digitized images taken from the portion of the specimen 
on the microscope slide in a second set of contiguous 
image tiles at a second, higher magnification; and 

addressing the first and second digitized images 
so that the higher magnification images can be easily 
located from the lower magnification images. 

2. The method of Claim 1 wherein the addressing 
step further comprises providing each of the image tiles 
in the first series and in the second series with 
coordinate information for enabling reconstruction of the 
first and second images. 

3. The method of Claim 1 wherein each of the 
digitized images of the series corresponds to 

25 substantially an optical view as seen by a person through 
an objective lens of the microscope. 

4. The method of Claim 1 including the step of 

remotely operating the controlled microscope to select a 

segment of the specimen from the first magnification view 

30 for digitally scanning and storing the second set of 
images tiles. 

5. A method of using a computer-controlled 
microscope imaging system for acquiring and storing 
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digitized microscopic images from a specimen on a 
microscope support, said method comprising the steps of: 

scanning and digitizing a specimen at a low 
magnification through a microscope to acquire digitized 
5 images ; 

displaying to the observer a low magnification, 
digitized image of the specimen; 

selecting a segment of the specimen from the low 
magnification digitized image for viewing at a higher 
10 magnification greater than the low magnification; 

scanning the selected segment at the higher 
magnification and digitizing the image of the segment to 
acquire digitized images; and 

storing the respective, low magnification, 
15 digitized images and the high magnification, digitized 
images of the segment. 

6. The method of Claim 5 wherein the step of 
scanning and digitizing at low magnification further 
includes the steps of : 

20 positioning and coordinating spatially a 

plurality of low magnification image tiles spatially to 
provide a composite macro image of the specimen formed 
from the plurality of low magnification image tiles; and 

providing a macro image of the specimens having 

25 a field of view substantially larger than the field of 
view yielded by the objective lens used for the low 
magnification scanning. 

7. A method in accordance with Claim 6 
including the step of reducing the size of the composite, 

30 macro image before displaying the composite macro image 
of the specimen to the viewer. 



8. A method in accordance with Claim 6 wherein 
the step of selecting and digitizing the low 
magnification, digitized image further includes the step 
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of positioning and coordinating spatially a plurality of 
low magnification images spatially to provide a composite 
image having a field of view substantially larger than 
the field of view yielded by the objective lens used for 
high magnification scanning. 

9. A method in accordance with Claim 5 in which 
the step of the selecting a defined segment area of 
interest further includes the step of moving a marker on 
the low magnification image to a point of interest to be 
viewed at high magnification; and operating the 
microscope to change objectives and conditions of 
illumination to provide a good image at high 
magnification. 

10. A method in accordance with Claim 9 wherein 
15 the high and low magnification images are each 

, reconstructed from image tiles each having a spatially 
known location used to assure their proper positioning in 
the reconstructed image. 
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11. A method in accordance with Claim 10 
wherein the step of automatically changing from one 
magnification scanning operation to another magnification 
scanning operation which includes one or more of the 
steps of: 

changing the microscope objective lens to a 
25 different magnification; 

changing at least one filter; 

changing the field diaphragm; 

changing the light intensity; 

changing the focus; and 
30 shifting the microscopic stage. 

12. The method of Claim 9 including the step of 
displaying the low magnification image of the specimen 
and displaying the marker on the low magnification image, 
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and simultaneously displaying the high magnification 
image of the segment at a position located by the marker 
on the low magnification image. 

13 . The method of Claim 5 including the step of 
5 shifting the marker to different segments and viewing at 
high magnification each of the selected segments; and 
marking on the low magnification image, each of the 
segments viewed at high magnification. 

14 . The method of Claim 5 including the steps 
10 of locating a computer-controlled microscope system used 
to make and store the digitized images on the Internet 
having a web site therefor; including the step of 
connecting an observer's computer and viewing screen over 
the Internet to the computer-controlled microscope at the 
15 web site in real time and transmitting the digitized 
signals for low magnification image and high 
magnification image from the web site to the observer's 
computer and viewing screen. 

15 - The method of Claim 5 including the step of 
20 enhancing the reconstructed, high magnification image by 
color filtering. 

16 . The method of Claim 5 including the step of 
providing a full color, low magnification, digitized 
image to the observer for aiding the observer in 

25 selecting a plurality of areas of interest on a priority 
basis. 

17. The method of Claim 5 including the steps 

of: 

analyzing the specimen on the slide with image 
30 analysis tests; and 
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reporting the results of the tests to the 
observer while at least one of the respective images is 
being viewed by the observer. 

18. The method of Claim 17 wherein the specimen 
5 is a biological specimen, and the analyzing step 

including performing analysis tests on tissue or cells in 
the biological specimen. 

19. A method in accordance with Claim 5 
including the step of the observer changing the 

10 magnification of the high magnification image to a ' 

magnification, intermediate the high and low magnification 
images by the use of an image analysis algorithm. 

20. A method in accordance with Claim 5 
including the step of and scrolling the high 

15 magnification image tiles to bring into view on a screen 
adjacent sections not previously viewed. 

21. A method of using a computer-controlled 
microscope imaging system for acquiring and analyzing 
areas of interest, said method comprising the steps of: 

scanning and digitizing a specimen at a low 
magnification through a microscope; 

displaying to the observer a low magnification, 
digitized image of the specimen; 

selecting a segment of the specimen from the low 
magnification digitized image for viewing at a higher 
magnification greater than the low magnification; 

automatically scanning the selected segment at 
the higher magnification and digitizing the image of the 
secfment ; and 

making available in real time to the view of the 
observer the respective, low magnification, digitized 
image and the high magnification, digitized image of the 
segment . 
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22. The method of Claim 21 wherein the step of 
scanning and digitizing at low magnification further 
includes the steps of : 

positioning and coordinating spatially a 
5 plurality of low magnification image tiles spatially to 
provide a composite macro image of the specimen formed 
from the plurality of low magnification image tiles; and 
providing a macro image of the specimens having 
a field of view substantially larger than the field of 
10 view yielded by the objective lens used for the low 
magnification scanning . 

23. A method of using digitized, microscope 
images of a specimen having images at a first resolution 
at a low magnification and images at a second resolution 

15 at a higher magnification, the method comprising the 
steps of: 

showing to a user a series of digitized, tiled 
and stored images abutted to form a macro image from the 
specimen at a first resolution ; 
20 selecting from the stored macro image an area 

thereon to the viewer to be viewed at higher resolution; 

showing to the user a series of abutted, 
digitized, tiled, stored images at the higher 
magnification and higher resolution to the viewer to be 
25 viewed at higher resolution. 

24. A method in accordance with Claim 23 
including the further step of shifting by user, back and 
forth between lower and higher resolution, stored images 
to analyze the specimen. 

30 25. A method in accordance with Claim 24 

including the further step of marking on the stored macro 
image the area to be viewed at higher resolution. 
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26. A method in accordance with Claim 23 
including the step of scrolling to neighboring, tiled 
images for viewing areas on the specimen adjacent the 
selected area. 



5 27. A method of acquiring and analyzing 

biological specimens with a computer- controlled 
microscope, said method comprising the steps of: 

scanning and digitizing a series of fields of 
views through a microscope objective lens of a low 
10 magnification and providing a series of digitized fields 
of view images ; 

arranging and positioning the adjacent fields of 
view images and forming therefrom a composite, macro 
image in color of the specimen having a field of view 
15 larger than that of the objective lens; 

displaying the macro color image to the observer 
to provide the observer with a larger field of view of 
the splecimen than the observer could see through an 
objective lens on the microscope; 
2° selecting a segment of the specimen from the 

macro image for viewing at a higher magnification than 
the low magnification; 

scanning and digitizing a series of fields of 
view through a microscope objective lens of a higher 
25 magnification and providing a series of digitized images; 

arranging and positioning the adjacent fields of 
view images at high magnification and forming therefrom a 
composite micro image of the segment having a field of 
view larger than that of the high magnification objective 
30 lens; and 

displaying to the observer both colored 
composite macro image and the micro image each having a 
field of view substantially larger than their respective 
objective lens. 
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28. A method in accordance with Claim 27 

including the step of marking on the macro image, the 

location of the micro image, so that the observer 

understands where the micro image is located on the macro 
5 image . 



29. A method in accordance with Claim 27 
including the step of selecting several segments each for 
viewing as a micro image and marking the location of each 
micro image on the composite image. 

10 30. A method in accordance with Claim 29 

including the step of recording the macro image with the 
marks thereon to provide a record of the segments viewed 
by the observer at high magnification to provide an audit 
trail for a later auditing of the segments viewed at 

15 higher resolution. 

31. A method of acquiring and analyzing 
biological specimens with a computer-controlled 
microscope; said method comprising the steps of: 

providing a biological specimen in position for 
20 scanning through the computer -control led microscope; 

scanning and digitizing the biological specimen 
through the microscope at a first, low magnification and 
acquiring and storing a digitized image of the specimen 
in color; 

25 displaying the stored, low magnification, 

digitized image in color to an observer to provide a 
macro image for viewing by the observer; 

selecting a segment of the colored, digitized, 
low magnification image for viewing at a higher 

30 magnification than the low magnification; 

scanning the segment at the higher magnification 
and acquiring a colored digitized image of the segment at 
the higher magnification to provide a micro image of the 
segment ; 
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making available in real time to the view of the 
observer, the respective micro and the macro images; and 

performing a quantitative analysis of the 
specimen from the digitized high magnification image . 

32. A method in accordance with Claim 3i 
including the step of marking on the macro image the 
position of the micro image so that the viewer 
understands the location of the micro view. 

33. A method in accordance with Claim 31 
10 including the steps of: 

digitizing a plurality of fields of view through 
the microscope at each of the low and high magnifications 
to form a series of digitized image tiles; 

arranging the respective image tiles to form the 
15 respective macro and micro images; 

displaying to the observer the micro and macro 
digitized images that each have a larger field of view 
than a single field of view through respective objective 
lens for low and high magnification. 

^° 34. A microscopic system for analyzing a 

specimen and a segment of the specimen, said apparatus 
comprising: 

a microscope having a plurality of objective 
lens to acquire images at different magnifications; 

a computer-controlled light illumination ' 
subsystem on the microscope for adjusting light 
illumination for different magnifications; 

a computer-controlled, focusing Subsystem on the 
microscope for adjusting the focus for different 
30 magnifications; 

a computer-controlled X and Y stage to position 
the specimen to be viewed at specified X and Y 
coordinates; 
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an imaging subsystem connected to a microscope 
to acquire and digitize the images of the specimen; 

a first optical system and image acquisition and 
image storing system having a first objective lens to 
5 acquire and arrange a stored series of images at a low 
magnification to provide a macro digitized image of the 
specimen having a field of view larger than the field of 
view from the objective lens used for the low 
magnification acquisition of images; 

10 a second optical and image acquisition and image 

storing system having a second, higher magnification 
objective lens to acquire and arrange a stored series of 
images at higher magnification and to provide a micro 
view image of the segment having a field of view larger 

15 than the field of view from the higher magnification 
objective lens. 

35. A system in accordance with Claim 34 
wherein the first image subsystem includes an X and Y 
position storage device for storing the X and Y 

20 coordinates for the macro image. 

36. A system in accordance with Claim 34 
wherein the second image acquisition subsystem has a 
subsystem for reconstructing high magnification image 
tiles with their X and Y coordinates positioned to 

25 reproduce spatially high resolution image tiles into the 
micro image as if the micro image was in the original 
image . 

37. A system in accordance with Claim 34 
wherein the first image acquisition subsystem has a 

30 subsystem for reconstructing low resolution image tiles 
with their X and Y coordinates positioned to reproduce 
spatially the low resolution tiles into a macro image of 
the specimen. 
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38. A microscopic system for analyzing a 
specimen and a segment of the specimen, said apparatus 
comprising: 

a microscope having a plurality of objective 
lens to acquire images at different magnifications; 

a computer- controlled light illumination 
subsystem on the microscope for adjusting light 
illumination for different magnifications; 

a computer-controlled, focusing subsystem on the 
microscope for adjusting the focus for different 
magnifications; 

a computer- control led X and y stage to position 
the specimen to be viewed at specified X and Y 
coordinates; 

an imaging subsystem connected to a microscope 
to acquire and digitize the images of the specimen; 

a first optical system and image acquisition 
system having a first objective lens to acquire and 
arrange a series of images at a low magnification to 
provide a macro digitized image of the specimen having a 
field of view larger than the field of view from the 
objective lens used for the low magnification acquisition 
of images ; 

a second optical and image acquisition system 
having a second, higher magnification objective lens to 
acquire and arrange a series of images at higher 
magnification and to provide a micro view image of the 
segment having a field of view larger than the field of 
view from the higher magnification objective lens; and 

a display screen device to make available in 
real time the micro and macro images on screens of the 
screen device. 



39. A system in accordance with Claim 38 
wherein the first image subsystem includes an X and Y 
35 position storage device for storing the X and Y 
coordinates for the macro image. 
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40. A method for analyzing biological specimens 
by an image analysis system having a computer- controlled, 
automated microscope comprising the following steps: 

placing a biological specimen in a microscope 
5 for viewing; 

using a computer terminal to control the image 
analysis system and microscope to acquire low 
magnification, image tiles of the specimen and to provide 
low magnification, composite, macro image from the 
10 plurality of image tiles; 

displaying the macro image on a screen of the 
computer texroinal; 

interactively selecting at least one point of 
interest on the displayed macro, specimen image for 
15 viewing at a higher magnification; 

sending signals from the computer terminal to 
control the image analysis system and the computer- 
controlled microscope to acquire a plurality of higher 
magnification, image tiles and to provide a low 
20 magnification, composite micro image; and 

analyzing the micro images of the specimen. 



41. A storage medium having digitized images of 
a specimen from a microscope slide comprising: 

a storage medium; 
25 a first series of contiguous, multiple digitized 

images at a first magnification abutted against each 
other to create an overall low resolution view of several 
adjacent, original microscope images assembled together; 

the first series of multiple, digitized images 
3 0 taken from at least a portion of a specimen on the slide; 

a second series of contiguous, multiple, 
digitized images at a second higher magnification abutted 
against each other to create a high resolution view of 
several adjacent, original microscope images assembled 
35 together, taken from said portion of the specimen; and 
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the first and second series of digitized images 
providing multiple resolution images of the slide 
specimen to a viewer, 

42. A storage medium in accordance with Claim 
5 41 wherein each of the series of contiguous, multiple 

images corresponds to an optical image seen by a person 
through an objective lens of a microscope. 

43. A storage medium in accordance with Claim 
41 wherein each of the series of contiguous, multiple 

10 images comprises reduced image of a corresponding 
original microscope image. 

44. A storage medium in accordance with Claim 
41 wherein the storage medium comprises a RAM. 

45. A storage medium in accordance with Claim 
15 41 wherein the storage medium comprises a hard disk. 

46. A storage medium in accordance with Claim 
41 wherein the stored images may be disseminated through 
a server to remote devices located on a communication 
channel . 
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